[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:02 dist-test-slave-zpfg krb5kdc[24849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:02 dist-test-slave-zpfg krb5kdc[24849](info): set up 2 sockets
Aug 12 05:25:02 dist-test-slave-zpfg krb5kdc[24849](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 05:25:03.141242 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/test-user.keytab.
W20260812 05:25:03.558547 24841 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.254s user 0.000s sys 0.002s
Aug 12 05:25:03 dist-test-slave-zpfg krb5kdc[24849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
W20260812 05:25:04.349084 24841 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.24.66.126: real 0.765s user 0.000s sys 0.004s
I20260812 05:25:04.349802 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39451
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:39451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:04.441443 24862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:04.441720 24862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:04.441761 24862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:04.443029 24862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:04.443110 24862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:04.443135 24862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:04.443156 24862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:04.443187 24862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:04.444803 24862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:39451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:04.445120 24862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:04.445384 24862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:04.451475 24871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:04.451514 24868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:04.451812 24869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:04.451737 24862 server_base.cc:1034] running on GCE node
I20260812 05:25:04.452263 24862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:04.452605 24862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:04.453828 24862 hybrid_clock.cc:648] HybridClock initialized: now 1786512304453877 us; error 112 us; skew 500 ppm
Aug 12 05:25:04 dist-test-slave-zpfg krb5kdc[24849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512304, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:04.456713 24862 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:04.457223 24862 webserver.cc:460] Webserver started at http://127.24.66.126:34235/ using document root <none> and password file <none>
I20260812 05:25:04.457473 24862 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:04.457546 24862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:04.457707 24862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:04.459053 24862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "9d3262fcfc034d1c929e94eb220366cb"
format_stamp: "Formatted at 2026-08-12 05:25:04 on dist-test-slave-zpfg"
server_key: "fcb53fcd3dd9b01e5f7d05873663c6cf"
server_key_iv: "674ec17b3d8e016849066cd6a3717569"
server_key_version: "encryptionkey@0"
I20260812 05:25:04.459558 24862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "9d3262fcfc034d1c929e94eb220366cb"
format_stamp: "Formatted at 2026-08-12 05:25:04 on dist-test-slave-zpfg"
server_key: "fcb53fcd3dd9b01e5f7d05873663c6cf"
server_key_iv: "674ec17b3d8e016849066cd6a3717569"
server_key_version: "encryptionkey@0"
I20260812 05:25:04.461242 24862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:04.462276 24878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:04.462589 24862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:04.462740 24862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "9d3262fcfc034d1c929e94eb220366cb"
format_stamp: "Formatted at 2026-08-12 05:25:04 on dist-test-slave-zpfg"
server_key: "fcb53fcd3dd9b01e5f7d05873663c6cf"
server_key_iv: "674ec17b3d8e016849066cd6a3717569"
server_key_version: "encryptionkey@0"
I20260812 05:25:04.462850 24862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:04.483228 24862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:04.485869 24862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:04.486064 24862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:04.497061 24862 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:39451
I20260812 05:25:04.497116 24932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:39451 every 8 connection(s)
I20260812 05:25:04.497668 24862 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:04.499004 24933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:04.502226 24933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb: Bootstrap starting.
I20260812 05:25:04.502918 24933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:04.503175 24933 log.cc:824] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:04.504045 24933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb: No bootstrap required, opened a new log
I20260812 05:25:04.506597 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 24862
I20260812 05:25:04.506708 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:04.507593 24841 external_mini_cluster.cc:1428] Setting key d69f15e717f39a3475572fad1c49ece5
I20260812 05:25:04.507946 24933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3262fcfc034d1c929e94eb220366cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39451 } }
I20260812 05:25:04.508102 24933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:04.508208 24933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3262fcfc034d1c929e94eb220366cb, State: Initialized, Role: FOLLOWER
I20260812 05:25:04.508380 24933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3262fcfc034d1c929e94eb220366cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39451 } }
I20260812 05:25:04.508493 24933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:04.508566 24933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:04.508632 24933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [term 0 FOLLOWER]: Advancing to term 1
Aug 12 05:25:04 dist-test-slave-zpfg krb5kdc[24849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:04.526947 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:04.508598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.24.66.126:39451)
0812 05:25:04.509074 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:04.509084 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:04.509270 (+ 186us) server_negotiation.cc:407] Connection header received
0812 05:25:04.512855 (+ 3585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:04.512866 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:04.512913 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:04.513055 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:04.515134 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:04.516187 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:04.517465 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:04.517912 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:04.518053 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:04.522513 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:04.522517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:04.522517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:04.522529 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:04.524867 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:04.525375 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:04.525378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:04.525379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:04.525488 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:04.525760 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:04.525761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:04.525761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:04.526031 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:04.526240 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:04.526589 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 05:25:04.526727 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":277,"threads_started":1}
I20260812 05:25:04.532626 24933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3262fcfc034d1c929e94eb220366cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39451 } }
I20260812 05:25:04.532836 24933 leader_election.cc:302] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d3262fcfc034d1c929e94eb220366cb; no voters:
I20260812 05:25:04.533054 24933 leader_election.cc:288] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:04.533317 24938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:04.533543 24933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:04.533587 24938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [term 1 LEADER]: Becoming Leader. State: Replica: 9d3262fcfc034d1c929e94eb220366cb, State: Running, Role: LEADER
I20260812 05:25:04.533896 24938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3262fcfc034d1c929e94eb220366cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39451 } }
I20260812 05:25:04.534906 24939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d3262fcfc034d1c929e94eb220366cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3262fcfc034d1c929e94eb220366cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39451 } } }
I20260812 05:25:04.535028 24939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:04.535270 24940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d3262fcfc034d1c929e94eb220366cb. Latest consensus state: current_term: 1 leader_uuid: "9d3262fcfc034d1c929e94eb220366cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d3262fcfc034d1c929e94eb220366cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39451 } } }
I20260812 05:25:04.535346 24940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:04.536072 24944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:04.537307 24944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:04.539477 24944 catalog_manager.cc:1347] Generated new cluster ID: 1f607564479d4d01a2bef1cd07769844
I20260812 05:25:04.540172 24944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:04.564332 24944 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:04.565093 24944 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:04.575210 24944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d3262fcfc034d1c929e94eb220366cb: Generated new TSK 0
I20260812 05:25:04.575493 24944 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
W20260812 05:25:04.785887 24841 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.24.66.65: real 0.201s user 0.000s sys 0.005s
I20260812 05:25:04.786325 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39451
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:04.902220 24959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:04.902506 24959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:04.902556 24959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:04.904103 24959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:04.904174 24959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:04.904208 24959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:04.906014 24959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39451
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:04.906281 24959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:04.906589 24959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:04.913623 24967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:04.913609 24966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:04.913623 24969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:04.914000 24959 server_base.cc:1034] running on GCE node
I20260812 05:25:04.914225 24959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:04.914522 24959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:04.915730 24959 hybrid_clock.cc:648] HybridClock initialized: now 1786512304915734 us; error 49 us; skew 500 ppm
Aug 12 05:25:04 dist-test-slave-zpfg krb5kdc[24849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512304, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:04.919363 24959 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:25:04.919833 24959 webserver.cc:460] Webserver started at http://127.24.66.65:43303/ using document root <none> and password file <none>
I20260812 05:25:04.920090 24959 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:04.920164 24959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:04.920382 24959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:04.921939 24959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "89ed981af5684efca437185cf5d2516a"
format_stamp: "Formatted at 2026-08-12 05:25:04 on dist-test-slave-zpfg"
server_key: "bfff6573dc4e0400ea3a6fe5f7977245"
server_key_iv: "b93638e87081a16f0a7535103da59bb3"
server_key_version: "encryptionkey@0"
I20260812 05:25:04.922449 24959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "89ed981af5684efca437185cf5d2516a"
format_stamp: "Formatted at 2026-08-12 05:25:04 on dist-test-slave-zpfg"
server_key: "bfff6573dc4e0400ea3a6fe5f7977245"
server_key_iv: "b93638e87081a16f0a7535103da59bb3"
server_key_version: "encryptionkey@0"
I20260812 05:25:04.924201 24959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 05:25:04.925239 24976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:04.925470 24959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:25:04.925578 24959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "89ed981af5684efca437185cf5d2516a"
format_stamp: "Formatted at 2026-08-12 05:25:04 on dist-test-slave-zpfg"
server_key: "bfff6573dc4e0400ea3a6fe5f7977245"
server_key_iv: "b93638e87081a16f0a7535103da59bb3"
server_key_version: "encryptionkey@0"
I20260812 05:25:04.925668 24959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:04.945761 24959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:04.948818 24959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:04.949139 24959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:04.949647 24959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:04.950078 24959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:04.950225 24959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:04.950277 24959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:04.950309 24959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:04.966516 24959 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:39095
I20260812 05:25:04.966581 25094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:39095 every 8 connection(s)
I20260812 05:25:04.967052 24959 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:04.974242 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 24959
I20260812 05:25:04.974382 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:04.974543 24841 external_mini_cluster.cc:1428] Setting key 95d54f59f6642e2ac01045cfddbd586f
Aug 12 05:25:04 dist-test-slave-zpfg krb5kdc[24849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512304, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:04.980608 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:04.967964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:42115 (local address 127.24.66.126:39451)
0812 05:25:04.968075 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:04.968079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:04.968669 (+ 590us) server_negotiation.cc:407] Connection header received
0812 05:25:04.969571 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:04.969572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:04.969587 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:04.969648 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:04.971089 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:04.971728 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:04.973039 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:04.973355 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:04.973430 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:04.976743 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:04.976745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:04.976745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:04.976754 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:04.978628 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:04.979268 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:04.979269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:04.979270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:04.979324 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:04.979658 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:04.979659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:04.979659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:04.979817 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:04.979875 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:04.980368 (+ 493us) server_negotiation.cc:299] Negotiation successful
0812 05:25:04.980474 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 05:25:04.980942 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:04.967953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39451 (local address 127.24.66.65:42115)
0812 05:25:04.968469 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:04.968525 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:04.969433 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:04.969726 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:04.969730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:04.970593 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:04.970967 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:04.970973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:04.971862 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:04.971864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:04.972921 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:04.972924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:04.973480 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:04.973481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:04.973710 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:04.974560 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:04.974587 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:04.976624 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:04.978768 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:04.978777 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:04.978780 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:04.979175 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:04.979403 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:04.979404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:04.979404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:04.979583 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:04.979894 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:04.979904 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:04.980149 (+ 245us) client_negotiation.cc:769] Sending connection context
0812 05:25:04.980291 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 05:25:04.980545 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":249,"threads_started":1}
I20260812 05:25:04.981815 25095 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39451
I20260812 05:25:04.981950 25095 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:04.982151 25095 heartbeater.cc:511] Master 127.24.66.126:39451 requested a full tablet report, sending...
I20260812 05:25:04.982854 24896 ts_manager.cc:194] Registered new tserver with Master: 89ed981af5684efca437185cf5d2516a (127.24.66.65:39095)
I20260812 05:25:04.983982 24896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:42115
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:25:05.007076 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39451
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:05.115666 25100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:05.115938 25100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:05.115967 25100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:05.117218 25100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:05.117264 25100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:05.117293 25100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:05.119063 25100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39451
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:05.119297 25100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:05.119491 25100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:05.125789 25106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:05.125901 25108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:05.125859 25105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:05.126122 25100 server_base.cc:1034] running on GCE node
I20260812 05:25:05.126523 25100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:05.126835 25100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:05.128044 25100 hybrid_clock.cc:648] HybridClock initialized: now 1786512305127989 us; error 78 us; skew 500 ppm
Aug 12 05:25:05 dist-test-slave-zpfg krb5kdc[24849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512305, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:05.131325 25100 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:25:05.131821 25100 webserver.cc:460] Webserver started at http://127.24.66.66:46811/ using document root <none> and password file <none>
I20260812 05:25:05.132081 25100 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:05.132133 25100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:05.132274 25100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:05.133704 25100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "4499da9da5794c39ad2538ecdf5c5e40"
format_stamp: "Formatted at 2026-08-12 05:25:05 on dist-test-slave-zpfg"
server_key: "1d82051663619f62d240d4691017dd82"
server_key_iv: "39a0e298b1a0e2b046f7e562fc03ecef"
server_key_version: "encryptionkey@0"
I20260812 05:25:05.134172 25100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "4499da9da5794c39ad2538ecdf5c5e40"
format_stamp: "Formatted at 2026-08-12 05:25:05 on dist-test-slave-zpfg"
server_key: "1d82051663619f62d240d4691017dd82"
server_key_iv: "39a0e298b1a0e2b046f7e562fc03ecef"
server_key_version: "encryptionkey@0"
I20260812 05:25:05.136456 25100 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 05:25:05.137444 25116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:05.137734 25100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:05.137852 25100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "4499da9da5794c39ad2538ecdf5c5e40"
format_stamp: "Formatted at 2026-08-12 05:25:05 on dist-test-slave-zpfg"
server_key: "1d82051663619f62d240d4691017dd82"
server_key_iv: "39a0e298b1a0e2b046f7e562fc03ecef"
server_key_version: "encryptionkey@0"
I20260812 05:25:05.137935 25100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:05.152030 25100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:05.154657 25100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:05.154809 25100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:05.155232 25100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:05.155579 25100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:05.155637 25100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:05.155673 25100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:05.155719 25100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:05.165594 25100 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:40729
I20260812 05:25:05.165740 25235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:40729 every 8 connection(s)
I20260812 05:25:05.166113 25100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:25:05 dist-test-slave-zpfg krb5kdc[24849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512305, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:05.174167 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 25100
I20260812 05:25:05.174248 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:05.174484 24841 external_mini_cluster.cc:1428] Setting key 37a82f3c494bb548f86afe433a3df7a8
I20260812 05:25:05.177491 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:05.167045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:60237 (local address 127.24.66.126:39451)
0812 05:25:05.167117 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:05.167120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:05.167430 (+ 310us) server_negotiation.cc:407] Connection header received
0812 05:25:05.168179 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:05.168180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:05.168192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:05.168246 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:05.169305 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.169808 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.170945 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.171140 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.171199 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:05.173994 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:05.173996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:05.173996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:05.174004 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:05.175819 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:05.176276 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:05.176278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:05.176278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:05.176349 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:05.176618 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:05.176619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:05.176620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:05.176742 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:05.176813 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:05.177256 (+ 443us) server_negotiation.cc:299] Negotiation successful
0812 05:25:05.177382 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:25:05.177944 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:05.167036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39451 (local address 127.24.66.66:60237)
0812 05:25:05.167288 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:05.167329 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:05.168079 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:05.168313 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:05.168316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:05.168959 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:05.169210 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:05.169216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.169916 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.169918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:05.170855 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:05.170858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.171391 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.171392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:05.171566 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:05.172323 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:05.172332 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:05.173864 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:05.175926 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:05.175935 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:05.175937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:05.176194 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:05.176420 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:05.176421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:05.176421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:05.176541 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:05.176824 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:05.176832 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:05.176998 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 05:25:05.177158 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 05:25:05.177414 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":151,"threads_started":1}
I20260812 05:25:05.178835 25236 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39451
I20260812 05:25:05.178952 25236 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:05.179163 25236 heartbeater.cc:511] Master 127.24.66.126:39451 requested a full tablet report, sending...
I20260812 05:25:05.179628 24896 ts_manager.cc:194] Registered new tserver with Master: 4499da9da5794c39ad2538ecdf5c5e40 (127.24.66.66:40729)
I20260812 05:25:05.180198 24896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:60237
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:25:05.206574 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39451
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:05.306384 25242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:05.306644 25242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:05.306674 25242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:05.308036 25242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:05.308087 25242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:05.308125 25242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:05.310003 25242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39451
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:05.310307 25242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:05.310644 25242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:05.317088 25250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:05.317055 25247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:05.317097 25248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:05.317464 25242 server_base.cc:1034] running on GCE node
I20260812 05:25:05.317672 25242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:05.317890 25242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:05.319048 25242 hybrid_clock.cc:648] HybridClock initialized: now 1786512305319034 us; error 29 us; skew 500 ppm
Aug 12 05:25:05 dist-test-slave-zpfg krb5kdc[24849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512305, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:05.322052 25242 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:25:05.322737 25242 webserver.cc:460] Webserver started at http://127.24.66.67:38921/ using document root <none> and password file <none>
I20260812 05:25:05.322999 25242 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:05.323073 25242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:05.323239 25242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:05.324509 25242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "4c6f358dfaac4ddab0efcd8494dba08e"
format_stamp: "Formatted at 2026-08-12 05:25:05 on dist-test-slave-zpfg"
server_key: "59bf77adf65c895c8e80f093f04501fe"
server_key_iv: "e85e1b4c7ea9318fbd0779c814fb58b6"
server_key_version: "encryptionkey@0"
I20260812 05:25:05.324904 25242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "4c6f358dfaac4ddab0efcd8494dba08e"
format_stamp: "Formatted at 2026-08-12 05:25:05 on dist-test-slave-zpfg"
server_key: "59bf77adf65c895c8e80f093f04501fe"
server_key_iv: "e85e1b4c7ea9318fbd0779c814fb58b6"
server_key_version: "encryptionkey@0"
I20260812 05:25:05.326526 25242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:05.327481 25257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:05.327688 25242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:05.327811 25242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "4c6f358dfaac4ddab0efcd8494dba08e"
format_stamp: "Formatted at 2026-08-12 05:25:05 on dist-test-slave-zpfg"
server_key: "59bf77adf65c895c8e80f093f04501fe"
server_key_iv: "e85e1b4c7ea9318fbd0779c814fb58b6"
server_key_version: "encryptionkey@0"
I20260812 05:25:05.327894 25242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:05.347690 25242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:05.349829 25242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:05.350005 25242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:05.350317 25242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:05.350724 25242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:05.350780 25242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:05.350822 25242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:05.350869 25242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:05.360152 25242 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:46883
I20260812 05:25:05.360206 25375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:46883 every 8 connection(s)
I20260812 05:25:05.360572 25242 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:05.363062 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 25242
I20260812 05:25:05.363149 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:05.363365 24841 external_mini_cluster.cc:1428] Setting key 73955d87dc76a376a4aadab9da6f2bd4
Aug 12 05:25:05 dist-test-slave-zpfg krb5kdc[24849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512305, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:05.371210 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:05.361418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:34399 (local address 127.24.66.126:39451)
0812 05:25:05.361505 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:05.361507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:05.361706 (+ 199us) server_negotiation.cc:407] Connection header received
0812 05:25:05.362382 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:05.362383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:05.362395 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:05.362438 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:05.363559 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.364274 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.365328 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.365601 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.365686 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:05.368083 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:05.368084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:05.368085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:05.368093 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:05.369741 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:05.370196 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:05.370198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:05.370198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:05.370260 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:05.370543 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:05.370543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:05.370544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:05.370681 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:05.370759 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:05.371036 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 05:25:05.371116 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:25:05.371533 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:05.361376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39451 (local address 127.24.66.67:34399)
0812 05:25:05.361568 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:05.361615 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:05.362240 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:05.362520 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:05.362524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:05.363194 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:05.363444 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:05.363450 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.364428 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.364430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:05.365229 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:05.365231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.365699 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.365701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:05.365843 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:05.366424 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:05.366433 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:05.367962 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:05.369855 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:05.369863 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:05.369865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:05.370090 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:05.370330 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:05.370331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:05.370332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:05.370465 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:05.370750 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:05.370757 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:05.370898 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 05:25:05.371012 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 05:25:05.371285 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260812 05:25:05.372325 25376 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39451
I20260812 05:25:05.372537 25376 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:05.372736 25376 heartbeater.cc:511] Master 127.24.66.126:39451 requested a full tablet report, sending...
I20260812 05:25:05.373250 24896 ts_manager.cc:194] Registered new tserver with Master: 4c6f358dfaac4ddab0efcd8494dba08e (127.24.66.67:46883)
I20260812 05:25:05.373728 24896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:34399
I20260812 05:25:05.375567 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 05:25:05 dist-test-slave-zpfg krb5kdc[24849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 05:25:05 dist-test-slave-zpfg krb5kdc[24849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:25:05.404068 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:05.394702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.24.66.65:39095)
0812 05:25:05.394878 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:05.394884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:05.394905 (+ 21us) server_negotiation.cc:407] Connection header received
0812 05:25:05.394934 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:05.394936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:05.394950 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:05.395022 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:05.395639 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.396509 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.397441 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.397920 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.397997 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:05.400744 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:05.400753 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:05.400753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:05.400762 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:05.402629 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:05.402976 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:05.402979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:05.402980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:05.403038 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:05.403401 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:05.403402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:05.403403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:05.403574 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:05.403815 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:05.403858 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 05:25:05.403947 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
W20260812 05:25:05.404670 25007 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57110
Aug 12 05:25:05 dist-test-slave-zpfg krb5kdc[24849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 05:25:05 dist-test-slave-zpfg krb5kdc[24849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:25:05.430277 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:05.422542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57122 (local address 127.24.66.65:39095)
0812 05:25:05.422614 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:05.422616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:05.422626 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:05.422700 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:05.422701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:05.422713 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:05.422765 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:05.423435 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.424033 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.424775 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:05.424974 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:05.425029 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:05.427621 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:05.427623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:05.427623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:05.427633 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:05.429053 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:05.429412 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:05.429413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:05.429413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:05.429473 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:05.429679 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:05.429679 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:05.429680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:05.429826 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:05.429951 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:05.430055 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 05:25:05.430144 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:25:05.431257 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 24959
I20260812 05:25:05.438678 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 25100
I20260812 05:25:05.444968 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 25242
I20260812 05:25:05.450750 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 24862
2026-08-12T05:25:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3498 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:05 dist-test-slave-zpfg krb5kdc[25391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:05 dist-test-slave-zpfg krb5kdc[25391](info): set up 2 sockets
Aug 12 05:25:05 dist-test-slave-zpfg krb5kdc[25391](info): commencing operation
krb5kdc: starting...
W20260812 05:25:06.563223 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:06 dist-test-slave-zpfg krb5kdc[25391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:25:06.701390 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:41105
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:35359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:41105
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:06.801084 25405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:06.801334 25405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:06.801360 25405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:06.802686 25405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:06.802738 25405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:06.802753 25405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:06.802767 25405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:06.802778 25405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:06.804458 25405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:41105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:41105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:06.804750 25405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:06.804999 25405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:06.810271 25410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:06.810272 25411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:06.810271 25413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:06.810309 25405 server_base.cc:1034] running on GCE node
I20260812 05:25:06.810786 25405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:06.811138 25405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:06.812331 25405 hybrid_clock.cc:648] HybridClock initialized: now 1786512306812312 us; error 43 us; skew 500 ppm
Aug 12 05:25:06 dist-test-slave-zpfg krb5kdc[25391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512306, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:06.815063 25405 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:06.815593 25405 webserver.cc:460] Webserver started at http://127.24.66.126:46481/ using document root <none> and password file <none>
I20260812 05:25:06.815805 25405 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:06.815847 25405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:06.815945 25405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:06.817013 25405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "1d852b45a7814a63b897797101868ad8"
format_stamp: "Formatted at 2026-08-12 05:25:06 on dist-test-slave-zpfg"
server_key: "9642112430427eb78ddd478ba1fe1255"
server_key_iv: "46582981d481afe017b07a44e6b2ff05"
server_key_version: "encryptionkey@0"
I20260812 05:25:06.817343 25405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "1d852b45a7814a63b897797101868ad8"
format_stamp: "Formatted at 2026-08-12 05:25:06 on dist-test-slave-zpfg"
server_key: "9642112430427eb78ddd478ba1fe1255"
server_key_iv: "46582981d481afe017b07a44e6b2ff05"
server_key_version: "encryptionkey@0"
I20260812 05:25:06.818943 25405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:06.819927 25420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:06.820322 25405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:06.820466 25405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "1d852b45a7814a63b897797101868ad8"
format_stamp: "Formatted at 2026-08-12 05:25:06 on dist-test-slave-zpfg"
server_key: "9642112430427eb78ddd478ba1fe1255"
server_key_iv: "46582981d481afe017b07a44e6b2ff05"
server_key_version: "encryptionkey@0"
I20260812 05:25:06.820576 25405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:06.847177 25405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:06.849709 25405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:06.849901 25405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:06.854630 25405 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:41105
I20260812 05:25:06.854770 25476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:41105 every 8 connection(s)
I20260812 05:25:06.855099 25405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:06.856169 25477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:06.858951 25477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8: Bootstrap starting.
I20260812 05:25:06.859313 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 25405
I20260812 05:25:06.859436 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:06.859599 24841 external_mini_cluster.cc:1428] Setting key bc683b0e1a68549da7f76da18bd4387f
I20260812 05:25:06.859779 25477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:06.860186 25477 log.cc:824] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:06.861016 25477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8: No bootstrap required, opened a new log
I20260812 05:25:06.863844 25477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852b45a7814a63b897797101868ad8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41105 } }
I20260812 05:25:06.864040 25477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:06.864077 25477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d852b45a7814a63b897797101868ad8, State: Initialized, Role: FOLLOWER
I20260812 05:25:06.864217 25477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852b45a7814a63b897797101868ad8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41105 } }
I20260812 05:25:06.864297 25477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:06.864331 25477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:06.864395 25477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:06.865223 25477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852b45a7814a63b897797101868ad8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41105 } }
I20260812 05:25:06.865357 25477 leader_election.cc:302] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d852b45a7814a63b897797101868ad8; no voters:
I20260812 05:25:06.865588 25477 leader_election.cc:288] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:06.865695 25482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:06.865947 25477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:06.865968 25482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [term 1 LEADER]: Becoming Leader. State: Replica: 1d852b45a7814a63b897797101868ad8, State: Running, Role: LEADER
I20260812 05:25:06.866173 25482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852b45a7814a63b897797101868ad8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41105 } }
I20260812 05:25:06.866583 25483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d852b45a7814a63b897797101868ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852b45a7814a63b897797101868ad8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41105 } } }
I20260812 05:25:06.866667 25483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:06.866845 25484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d852b45a7814a63b897797101868ad8. Latest consensus state: current_term: 1 leader_uuid: "1d852b45a7814a63b897797101868ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d852b45a7814a63b897797101868ad8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41105 } } }
I20260812 05:25:06.866904 25484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8 [sys.catalog]: This master's current role is: LEADER
Aug 12 05:25:06 dist-test-slave-zpfg krb5kdc[25391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
W20260812 05:25:06.868181 25499 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:25:06.868249 25499 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:25:06.868338 25490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:06.868528 25490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:06.870419 25490 catalog_manager.cc:1347] Generated new cluster ID: 708437e8dcbe4484a3846adc633553d5
I20260812 05:25:06.870474 25490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:06.872678 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:06.860229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41270 (local address 127.24.66.126:41105)
0812 05:25:06.860436 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:06.860448 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:06.860486 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:25:06.861065 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:06.861073 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:06.861159 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:06.861310 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:06.862217 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:06.863229 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:06.864446 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:06.864896 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:06.864989 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:06.868497 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:06.868501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:06.868501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:06.868515 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:06.870880 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:06.871356 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:06.871358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:06.871359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:06.871460 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:06.871698 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:06.871699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:06.871699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:06.871988 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:06.872137 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:06.872291 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 05:25:06.872431 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260812 05:25:06.884299 25490 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:06.884938 25490 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:06.887995 25490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d852b45a7814a63b897797101868ad8: Generated new TSK 0
I20260812 05:25:06.888230 25490 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:25:06.923516 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41105
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:35359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:07.014842 25504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:07.015085 25504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:07.015115 25504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:07.016372 25504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:07.016413 25504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:07.016441 25504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:07.017961 25504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:07.018186 25504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:07.018446 25504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:07.023975 25512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:07.024088 25509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:07.023962 25510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:07.024258 25504 server_base.cc:1034] running on GCE node
I20260812 05:25:07.024653 25504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:07.024946 25504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:07.026124 25504 hybrid_clock.cc:648] HybridClock initialized: now 1786512307026114 us; error 31 us; skew 500 ppm
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512307, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:07.035990 25504 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:25:07.036531 25504 webserver.cc:460] Webserver started at http://127.24.66.65:46725/ using document root <none> and password file <none>
I20260812 05:25:07.036782 25504 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:07.036832 25504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:07.036945 25504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:07.038169 25504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "59d73780503d4b818bd0dd8fbb79a71d"
format_stamp: "Formatted at 2026-08-12 05:25:07 on dist-test-slave-zpfg"
server_key: "cc1b528e1745460030042585c9ac620d"
server_key_iv: "3b9c136f9aef9c397c02db1c81c5acf7"
server_key_version: "encryptionkey@0"
I20260812 05:25:07.038605 25504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "59d73780503d4b818bd0dd8fbb79a71d"
format_stamp: "Formatted at 2026-08-12 05:25:07 on dist-test-slave-zpfg"
server_key: "cc1b528e1745460030042585c9ac620d"
server_key_iv: "3b9c136f9aef9c397c02db1c81c5acf7"
server_key_version: "encryptionkey@0"
I20260812 05:25:07.040201 25504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:07.041076 25519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:07.041296 25504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:07.041388 25504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "59d73780503d4b818bd0dd8fbb79a71d"
format_stamp: "Formatted at 2026-08-12 05:25:07 on dist-test-slave-zpfg"
server_key: "cc1b528e1745460030042585c9ac620d"
server_key_iv: "3b9c136f9aef9c397c02db1c81c5acf7"
server_key_version: "encryptionkey@0"
I20260812 05:25:07.041450 25504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:07.060707 25504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:07.063452 25504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:07.063581 25504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:07.063796 25504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:07.064067 25504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:07.064103 25504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:07.064128 25504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:07.064141 25504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:07.070798 25504 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:35857
I20260812 05:25:07.070919 25637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:35857 every 8 connection(s)
I20260812 05:25:07.071250 25504 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512307, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:07.079896 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 25504
I20260812 05:25:07.080003 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:07.080304 24841 external_mini_cluster.cc:1428] Setting key e63178a43d6f6c2a1a2e0fafe3864827
I20260812 05:25:07.081902 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.072075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:60203 (local address 127.24.66.126:41105)
0812 05:25:07.072137 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.072139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.072337 (+ 198us) server_negotiation.cc:407] Connection header received
0812 05:25:07.073398 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.073399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.073409 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.073479 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:07.074173 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.074740 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.075796 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.076020 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.076074 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.078538 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:07.078540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:07.078540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:07.078547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:07.080215 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.080797 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.080797 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.080798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.080852 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.081151 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.081152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.081152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.081302 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:07.081379 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.081651 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.081750 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 05:25:07.081925 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.072021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:41105 (local address 127.24.66.65:60203)
0812 05:25:07.072197 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:07.072243 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:07.073290 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:07.073578 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:07.073582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:07.073877 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:07.074085 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.074090 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.074847 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.074848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.075708 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.075711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.076117 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.076118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.076326 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.076966 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:07.076975 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:07.078452 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:07.080322 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:07.080329 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:07.080331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:07.080718 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:07.080917 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:07.080918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:07.080918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:07.081078 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:07.081377 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:07.081384 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:07.081513 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 05:25:07.081604 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 05:25:07.081742 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260812 05:25:07.083334 25638 heartbeater.cc:347] Connected to a master server at 127.24.66.126:41105
I20260812 05:25:07.083458 25638 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:07.083648 25638 heartbeater.cc:511] Master 127.24.66.126:41105 requested a full tablet report, sending...
I20260812 05:25:07.084213 25440 ts_manager.cc:194] Registered new tserver with Master: 59d73780503d4b818bd0dd8fbb79a71d (127.24.66.65:35857)
I20260812 05:25:07.084852 25440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:60203
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:25:07.109786 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41105
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:35359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:07.197311 25643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:07.197588 25643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:07.197629 25643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:07.198908 25643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:07.198982 25643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:07.199026 25643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:07.200635 25643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:07.200935 25643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:07.201206 25643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:07.206764 25649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:07.206923 25648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:07.206822 25651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:07.207134 25643 server_base.cc:1034] running on GCE node
I20260812 05:25:07.207387 25643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:07.207625 25643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:07.208833 25643 hybrid_clock.cc:648] HybridClock initialized: now 1786512307208810 us; error 54 us; skew 500 ppm
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512307, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:07.211943 25643 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:25:07.212446 25643 webserver.cc:460] Webserver started at http://127.24.66.66:41255/ using document root <none> and password file <none>
I20260812 05:25:07.212698 25643 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:07.212747 25643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:07.212854 25643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:07.214076 25643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "b15990f70a46414da43f765dfb9cb2cc"
format_stamp: "Formatted at 2026-08-12 05:25:07 on dist-test-slave-zpfg"
server_key: "22d735ca8dffc6c1acc5f7ffcc42eb5f"
server_key_iv: "f72b660946f624874d14d696e913da2d"
server_key_version: "encryptionkey@0"
I20260812 05:25:07.214520 25643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "b15990f70a46414da43f765dfb9cb2cc"
format_stamp: "Formatted at 2026-08-12 05:25:07 on dist-test-slave-zpfg"
server_key: "22d735ca8dffc6c1acc5f7ffcc42eb5f"
server_key_iv: "f72b660946f624874d14d696e913da2d"
server_key_version: "encryptionkey@0"
I20260812 05:25:07.216037 25643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:07.216987 25658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:07.217202 25643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:07.217298 25643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "b15990f70a46414da43f765dfb9cb2cc"
format_stamp: "Formatted at 2026-08-12 05:25:07 on dist-test-slave-zpfg"
server_key: "22d735ca8dffc6c1acc5f7ffcc42eb5f"
server_key_iv: "f72b660946f624874d14d696e913da2d"
server_key_version: "encryptionkey@0"
I20260812 05:25:07.217375 25643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:07.235925 25643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:07.238562 25643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:07.238700 25643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:07.238926 25643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:07.239233 25643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:07.239269 25643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:07.239295 25643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:07.239308 25643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:07.246019 25643 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:39415
I20260812 05:25:07.246065 25776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:39415 every 8 connection(s)
I20260812 05:25:07.246510 25643 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512307, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:07.256342 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 25643
I20260812 05:25:07.256457 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:07.256641 24841 external_mini_cluster.cc:1428] Setting key 08fd1fe0a7d5eceb86efddd5e668c175
I20260812 05:25:07.257843 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.247427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:58563 (local address 127.24.66.126:41105)
0812 05:25:07.247497 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.247499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.247704 (+ 205us) server_negotiation.cc:407] Connection header received
0812 05:25:07.248804 (+ 1100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.248806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.248818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.248876 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:07.249566 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.250076 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.251269 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.251476 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.251536 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.254191 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:07.254193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:07.254193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:07.254203 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:07.255936 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.256545 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.256546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.256547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.256665 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.257021 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.257028 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.257028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.257241 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:07.257341 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.257651 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.257747 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:25:07.258080 25779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.247384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:41105 (local address 127.24.66.66:58563)
0812 05:25:07.247582 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:07.247624 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:07.248676 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:07.248947 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:07.248950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:07.249232 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:07.249458 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.249462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.250183 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.250184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.251164 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.251167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.251568 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.251569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.251717 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.252384 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:07.252393 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:07.254077 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:07.256019 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:07.256026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:07.256027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:07.256431 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:07.256780 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:07.256781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:07.256782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:07.256961 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:07.257332 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:07.257340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:07.257500 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 05:25:07.257631 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 05:25:07.257808 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":118,"threads_started":1}
I20260812 05:25:07.258610 25777 heartbeater.cc:347] Connected to a master server at 127.24.66.126:41105
I20260812 05:25:07.258714 25777 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:07.258934 25777 heartbeater.cc:511] Master 127.24.66.126:41105 requested a full tablet report, sending...
I20260812 05:25:07.259397 25440 ts_manager.cc:194] Registered new tserver with Master: b15990f70a46414da43f765dfb9cb2cc (127.24.66.66:39415)
I20260812 05:25:07.260025 25440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:58563
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:25:07.290180 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41105
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:35359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:07.382022 25783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:07.382268 25783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:07.382296 25783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:07.383584 25783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:07.383637 25783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:07.383665 25783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:07.385659 25783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41105
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:07.385982 25783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:07.386281 25783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:07.392518 25792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:07.392519 25790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:07.392762 25783 server_base.cc:1034] running on GCE node
W20260812 05:25:07.392518 25789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:07.393157 25783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:07.393487 25783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:07.394671 25783 hybrid_clock.cc:648] HybridClock initialized: now 1786512307394659 us; error 31 us; skew 500 ppm
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512307, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:07.397924 25783 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:25:07.398526 25783 webserver.cc:460] Webserver started at http://127.24.66.67:42911/ using document root <none> and password file <none>
I20260812 05:25:07.398787 25783 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:07.398837 25783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:07.398941 25783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:07.400094 25783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "95c279a9181141f3afd8aa8626adc558"
format_stamp: "Formatted at 2026-08-12 05:25:07 on dist-test-slave-zpfg"
server_key: "bfa0d08ea4474617b4b20e940dd68ace"
server_key_iv: "090af814600a3a4c443461c8b55f1889"
server_key_version: "encryptionkey@0"
I20260812 05:25:07.400439 25783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "95c279a9181141f3afd8aa8626adc558"
format_stamp: "Formatted at 2026-08-12 05:25:07 on dist-test-slave-zpfg"
server_key: "bfa0d08ea4474617b4b20e940dd68ace"
server_key_iv: "090af814600a3a4c443461c8b55f1889"
server_key_version: "encryptionkey@0"
I20260812 05:25:07.401981 25783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:25:07.402940 25800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:07.403138 25783 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:25:07.403235 25783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "95c279a9181141f3afd8aa8626adc558"
format_stamp: "Formatted at 2026-08-12 05:25:07 on dist-test-slave-zpfg"
server_key: "bfa0d08ea4474617b4b20e940dd68ace"
server_key_iv: "090af814600a3a4c443461c8b55f1889"
server_key_version: "encryptionkey@0"
I20260812 05:25:07.403298 25783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:07.426998 25783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:07.429387 25783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:07.429514 25783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:07.429730 25783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:07.430085 25783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:07.430122 25783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:07.430146 25783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:07.430159 25783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:07.439472 25783 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:45003
I20260812 05:25:07.439551 25918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:45003 every 8 connection(s)
I20260812 05:25:07.440049 25783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:07.446720 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 25783
I20260812 05:25:07.446846 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:07.447031 24841 external_mini_cluster.cc:1428] Setting key 958afaa48e6d6c3d9e9824be27fca0e4
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512307, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:07.451910 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.441235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:33243 (local address 127.24.66.126:41105)
0812 05:25:07.441399 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.441401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.441630 (+ 229us) server_negotiation.cc:407] Connection header received
0812 05:25:07.442912 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.442913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.442924 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.442991 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:07.443809 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.444410 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.445735 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.445964 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.446026 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.448910 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:07.448912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:07.448912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:07.448919 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:07.450311 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.450796 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.450797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.450797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.450884 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.451250 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.451251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.451251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.451379 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:07.451434 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.451738 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.451806 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260812 05:25:07.452308 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.441148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:41105 (local address 127.24.66.67:33243)
0812 05:25:07.441484 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:07.441527 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:07.442808 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:07.443055 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:07.443058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:07.443439 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:07.443689 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.443695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.444506 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.444507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.445649 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.445652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.446057 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.446057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.446221 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.447038 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:07.447049 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:07.448823 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:07.450458 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:07.450466 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:07.450467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:07.450727 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:07.450974 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:07.450975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:07.450975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:07.451112 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:07.451457 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:07.451463 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:07.451601 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 05:25:07.451723 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 05:25:07.451888 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":91,"threads_started":1}
I20260812 05:25:07.453019 25919 heartbeater.cc:347] Connected to a master server at 127.24.66.126:41105
I20260812 05:25:07.453280 25919 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:07.453513 25919 heartbeater.cc:511] Master 127.24.66.126:41105 requested a full tablet report, sending...
I20260812 05:25:07.454022 25440 ts_manager.cc:194] Registered new tserver with Master: 95c279a9181141f3afd8aa8626adc558 (127.24.66.67:45003)
I20260812 05:25:07.454499 25440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:33243
I20260812 05:25:07.459022 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:25:07.467892 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.461199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39794 (local address 127.24.66.126:41105)
0812 05:25:07.461289 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.461291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.461300 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:07.461323 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.461324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.461335 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.461389 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:07.462101 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.462690 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.463469 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.463648 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.463699 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.465088 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:07.465089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:07.465090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:07.465095 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:07.466592 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.466986 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.466987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.466988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.467047 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.467345 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.467346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.467346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.467511 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:07.467594 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.467752 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.467822 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 05:25:07.470965 25440 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:07.472803 25440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:07.480433 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.475671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:45003 (local address 127.0.0.1:35024)
0812 05:25:07.476323 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:07.476333 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:07.476414 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:07.476721 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:07.476722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:07.476724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:07.476790 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.476791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.477750 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.477751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.479074 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.479077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.480096 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.480098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.480200 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.480221 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 05:25:07.480290 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:25:07.480348 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":115,"threads_started":1}
I20260812 05:25:07.480463 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.475949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.24.66.67:45003)
0812 05:25:07.476261 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.476266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.476464 (+ 198us) server_negotiation.cc:407] Connection header received
0812 05:25:07.476513 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.476515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.476533 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.476619 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:07.476872 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.477590 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.479213 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.479983 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.480197 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.480276 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.480308 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.480349 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":87,"threads_started":1}
I20260812 05:25:07.480969 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.475980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46520 (local address 127.24.66.65:35857)
0812 05:25:07.476189 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.476193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.476652 (+ 459us) server_negotiation.cc:407] Connection header received
0812 05:25:07.476737 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.476740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.476755 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.476829 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:07.476994 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.478065 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.479597 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.480438 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.480499 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.480559 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.480832 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.480880 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":89,"threads_started":1}
I20260812 05:25:07.481277 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.475672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:35857 (local address 127.0.0.1:46520)
0812 05:25:07.476517 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:07.476532 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:07.476580 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:07.476839 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:07.476840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:07.476841 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:07.476873 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.476874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.478249 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.478251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.479477 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.479481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.480592 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.480593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.480646 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.480649 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:07.480698 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 05:25:07.481154 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":2,"thread_start_us":112,"threads_started":1}
I20260812 05:25:07.481356 25932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.475668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:39415 (local address 127.0.0.1:48072)
0812 05:25:07.475795 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:07.475836 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:07.475927 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:07.476471 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:07.476475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:07.476478 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:07.476579 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.476581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.478065 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.478066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.479335 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.479337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.480407 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.480415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.480487 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.480492 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:25:07.480524 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 05:25:07.481217 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":78,"threads_started":1}
I20260812 05:25:07.481592 25938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.476015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.24.66.66:39415)
0812 05:25:07.476265 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.476271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.476282 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:07.476338 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.476340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.476358 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.476675 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:07.476781 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.477947 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.479428 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.480280 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.480993 (+ 713us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.481074 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.481100 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.481494 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":176,"threads_started":1}
I20260812 05:25:07.481745 25850 tablet_service.cc:1467] Processing CreateTablet for tablet f5511357fd3d49b8906991c9e942874b (DEFAULT_TABLE table=test-table [id=86b1cb0f708d45a7aefeed339690d068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:07.482012 25569 tablet_service.cc:1467] Processing CreateTablet for tablet f5511357fd3d49b8906991c9e942874b (DEFAULT_TABLE table=test-table [id=86b1cb0f708d45a7aefeed339690d068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:07.482187 25850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5511357fd3d49b8906991c9e942874b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:07.482282 25569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5511357fd3d49b8906991c9e942874b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:07.482261 25707 tablet_service.cc:1467] Processing CreateTablet for tablet f5511357fd3d49b8906991c9e942874b (DEFAULT_TABLE table=test-table [id=86b1cb0f708d45a7aefeed339690d068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:07.482607 25707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5511357fd3d49b8906991c9e942874b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:07.485663 25939 tablet_bootstrap.cc:492] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d: Bootstrap starting.
I20260812 05:25:07.486851 25940 tablet_bootstrap.cc:492] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558: Bootstrap starting.
I20260812 05:25:07.486997 25939 tablet_bootstrap.cc:654] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:07.488227 25939 log.cc:824] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:07.488304 25940 tablet_bootstrap.cc:654] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:07.488936 25942 tablet_bootstrap.cc:492] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc: Bootstrap starting.
I20260812 05:25:07.489341 25940 log.cc:824] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:07.489581 25939 tablet_bootstrap.cc:492] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d: No bootstrap required, opened a new log
I20260812 05:25:07.489666 25939 ts_tablet_manager.cc:1397] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 05:25:07.490302 25942 tablet_bootstrap.cc:654] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:07.490677 25940 tablet_bootstrap.cc:492] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558: No bootstrap required, opened a new log
I20260812 05:25:07.490793 25940 ts_tablet_manager.cc:1397] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 05:25:07.491161 25942 log.cc:824] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:07.491922 25942 tablet_bootstrap.cc:492] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc: No bootstrap required, opened a new log
I20260812 05:25:07.492010 25942 ts_tablet_manager.cc:1397] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 05:25:07.493626 25939 raft_consensus.cc:348] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c279a9181141f3afd8aa8626adc558" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 45003 } } peers { permanent_uuid: "59d73780503d4b818bd0dd8fbb79a71d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35857 } } peers { permanent_uuid: "b15990f70a46414da43f765dfb9cb2cc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 39415 } }
I20260812 05:25:07.493947 25939 raft_consensus.cc:374] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:07.493993 25939 raft_consensus.cc:729] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59d73780503d4b818bd0dd8fbb79a71d, State: Initialized, Role: FOLLOWER
I20260812 05:25:07.494185 25939 consensus_queue.cc:260] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c279a9181141f3afd8aa8626adc558" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 45003 } } peers { permanent_uuid: "59d73780503d4b818bd0dd8fbb79a71d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35857 } } peers { permanent_uuid: "b15990f70a46414da43f765dfb9cb2cc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 39415 } }
I20260812 05:25:07.494406 25940 raft_consensus.cc:348] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c279a9181141f3afd8aa8626adc558" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 45003 } } peers { permanent_uuid: "59d73780503d4b818bd0dd8fbb79a71d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35857 } } peers { permanent_uuid: "b15990f70a46414da43f765dfb9cb2cc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 39415 } }
I20260812 05:25:07.494711 25638 heartbeater.cc:503] Master 127.24.66.126:41105 was elected leader, sending a full tablet report...
I20260812 05:25:07.494704 25940 raft_consensus.cc:374] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:07.494791 25939 ts_tablet_manager.cc:1428] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260812 05:25:07.494786 25940 raft_consensus.cc:729] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c279a9181141f3afd8aa8626adc558, State: Initialized, Role: FOLLOWER
I20260812 05:25:07.494938 25940 consensus_queue.cc:260] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c279a9181141f3afd8aa8626adc558" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 45003 } } peers { permanent_uuid: "59d73780503d4b818bd0dd8fbb79a71d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35857 } } peers { permanent_uuid: "b15990f70a46414da43f765dfb9cb2cc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 39415 } }
I20260812 05:25:07.495317 25919 heartbeater.cc:503] Master 127.24.66.126:41105 was elected leader, sending a full tablet report...
I20260812 05:25:07.495343 25940 ts_tablet_manager.cc:1428] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260812 05:25:07.496066 25942 raft_consensus.cc:348] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c279a9181141f3afd8aa8626adc558" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 45003 } } peers { permanent_uuid: "59d73780503d4b818bd0dd8fbb79a71d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35857 } } peers { permanent_uuid: "b15990f70a46414da43f765dfb9cb2cc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 39415 } }
I20260812 05:25:07.496369 25942 raft_consensus.cc:374] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:07.496433 25942 raft_consensus.cc:729] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15990f70a46414da43f765dfb9cb2cc, State: Initialized, Role: FOLLOWER
I20260812 05:25:07.496555 25942 consensus_queue.cc:260] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c279a9181141f3afd8aa8626adc558" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 45003 } } peers { permanent_uuid: "59d73780503d4b818bd0dd8fbb79a71d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35857 } } peers { permanent_uuid: "b15990f70a46414da43f765dfb9cb2cc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 39415 } }
I20260812 05:25:07.497191 25942 ts_tablet_manager.cc:1428] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260812 05:25:07.497187 25777 heartbeater.cc:503] Master 127.24.66.126:41105 was elected leader, sending a full tablet report...
W20260812 05:25:07.572108 25639 tablet.cc:2369] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:25:07.691123 25920 tablet.cc:2369] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:25:07.701920 25947 raft_consensus.cc:482] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:07.702062 25947 raft_consensus.cc:504] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c279a9181141f3afd8aa8626adc558" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 45003 } } peers { permanent_uuid: "59d73780503d4b818bd0dd8fbb79a71d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35857 } } peers { permanent_uuid: "b15990f70a46414da43f765dfb9cb2cc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 39415 } }
I20260812 05:25:07.702838 25947 leader_election.cc:288] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59d73780503d4b818bd0dd8fbb79a71d (127.24.66.65:35857), b15990f70a46414da43f765dfb9cb2cc (127.24.66.66:39415)
I20260812 05:25:07.706009 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.703059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:39415 (local address 127.24.66.67:43353)
0812 05:25:07.703118 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:07.703130 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:07.703176 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:07.703426 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:07.703427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:07.703428 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:07.703565 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.703567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.704226 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.704227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.705029 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.705031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.705735 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.705735 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.705819 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.705827 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 05:25:07.705893 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 05:25:07.705937 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 05:25:07.706009 25938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.703124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:43353 (local address 127.24.66.66:39415)
0812 05:25:07.703299 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.703300 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.703308 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:07.703332 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.703333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.703342 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.703404 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:07.703653 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.704135 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.705210 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.705645 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.705727 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.705736 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.705899 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.705938 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:25:07.706650 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5511357fd3d49b8906991c9e942874b" candidate_uuid: "95c279a9181141f3afd8aa8626adc558" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b15990f70a46414da43f765dfb9cb2cc" is_pre_election: true
I20260812 05:25:07.706874 25727 raft_consensus.cc:2454] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95c279a9181141f3afd8aa8626adc558 in term 0.
I20260812 05:25:07.707175 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.703108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:35857 (local address 127.24.66.67:47129)
0812 05:25:07.703392 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:07.703401 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:07.703481 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:07.703643 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:07.703643 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:07.703645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:07.703848 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.703850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.704699 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.704700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.706162 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:07.706166 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.706893 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.706895 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:07.706973 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.706979 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:25:07.707061 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 05:25:07.707110 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":185,"threads_started":1}
I20260812 05:25:07.707216 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.703124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:47129 (local address 127.24.66.65:35857)
0812 05:25:07.703184 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.703186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.703466 (+ 280us) server_negotiation.cc:407] Connection header received
0812 05:25:07.703546 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.703547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.703559 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.703625 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:07.703927 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.704478 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.706355 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.706737 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.706833 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.706842 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.707072 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.707150 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:25:07.707269 25801 leader_election.cc:302] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95c279a9181141f3afd8aa8626adc558, b15990f70a46414da43f765dfb9cb2cc; no voters:
I20260812 05:25:07.707491 25947 raft_consensus.cc:2781] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:07.707581 25947 raft_consensus.cc:482] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:07.707615 25947 raft_consensus.cc:3037] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:07.707721 25590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5511357fd3d49b8906991c9e942874b" candidate_uuid: "95c279a9181141f3afd8aa8626adc558" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59d73780503d4b818bd0dd8fbb79a71d" is_pre_election: true
I20260812 05:25:07.707981 25590 raft_consensus.cc:2454] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95c279a9181141f3afd8aa8626adc558 in term 0.
I20260812 05:25:07.708838 25947 raft_consensus.cc:504] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c279a9181141f3afd8aa8626adc558" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 45003 } } peers { permanent_uuid: "59d73780503d4b818bd0dd8fbb79a71d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35857 } } peers { permanent_uuid: "b15990f70a46414da43f765dfb9cb2cc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 39415 } }
I20260812 05:25:07.709010 25947 leader_election.cc:288] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [CANDIDATE]: Term 1 election: Requested vote from peers 59d73780503d4b818bd0dd8fbb79a71d (127.24.66.65:35857), b15990f70a46414da43f765dfb9cb2cc (127.24.66.66:39415)
I20260812 05:25:07.709226 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5511357fd3d49b8906991c9e942874b" candidate_uuid: "95c279a9181141f3afd8aa8626adc558" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b15990f70a46414da43f765dfb9cb2cc"
I20260812 05:25:07.709345 25590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5511357fd3d49b8906991c9e942874b" candidate_uuid: "95c279a9181141f3afd8aa8626adc558" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59d73780503d4b818bd0dd8fbb79a71d"
I20260812 05:25:07.709344 25727 raft_consensus.cc:3037] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:07.709425 25590 raft_consensus.cc:3037] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:07.710728 25590 raft_consensus.cc:2454] T f5511357fd3d49b8906991c9e942874b P 59d73780503d4b818bd0dd8fbb79a71d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95c279a9181141f3afd8aa8626adc558 in term 1.
I20260812 05:25:07.710728 25727 raft_consensus.cc:2454] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95c279a9181141f3afd8aa8626adc558 in term 1.
I20260812 05:25:07.710986 25804 leader_election.cc:302] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59d73780503d4b818bd0dd8fbb79a71d, 95c279a9181141f3afd8aa8626adc558; no voters:
I20260812 05:25:07.711150 25947 raft_consensus.cc:2781] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:07.711364 25947 raft_consensus.cc:686] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [term 1 LEADER]: Becoming Leader. State: Replica: 95c279a9181141f3afd8aa8626adc558, State: Running, Role: LEADER
I20260812 05:25:07.711472 25947 consensus_queue.cc:237] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c279a9181141f3afd8aa8626adc558" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 45003 } } peers { permanent_uuid: "59d73780503d4b818bd0dd8fbb79a71d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35857 } } peers { permanent_uuid: "b15990f70a46414da43f765dfb9cb2cc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 39415 } }
I20260812 05:25:07.712446 25440 catalog_manager.cc:5495] T f5511357fd3d49b8906991c9e942874b P 95c279a9181141f3afd8aa8626adc558 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95c279a9181141f3afd8aa8626adc558 (127.24.66.67). New cstate: current_term: 1 leader_uuid: "95c279a9181141f3afd8aa8626adc558" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95c279a9181141f3afd8aa8626adc558" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 45003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59d73780503d4b818bd0dd8fbb79a71d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b15990f70a46414da43f765dfb9cb2cc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 39415 } health_report { overall_health: UNKNOWN } } }
W20260812 05:25:07.747390 25778 tablet.cc:2369] T f5511357fd3d49b8906991c9e942874b P b15990f70a46414da43f765dfb9cb2cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:25:07.776085 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.767477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46526 (local address 127.24.66.65:35857)
0812 05:25:07.767569 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.767571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.767695 (+ 124us) server_negotiation.cc:407] Connection header received
0812 05:25:07.767838 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.767840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.767851 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.767904 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:07.768687 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.769198 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.769963 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.770166 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.770264 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.772860 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:07.772878 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:07.772879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:07.772892 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:07.774760 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.775161 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.775165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.775166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.775238 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.775478 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.775478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.775479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.775632 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:07.775742 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.775860 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.775937 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:25:07.803915 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.795395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46532 (local address 127.24.66.65:35857)
0812 05:25:07.795496 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.795499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.795600 (+ 101us) server_negotiation.cc:407] Connection header received
0812 05:25:07.795631 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.795632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.795644 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.795709 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:07.796307 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.796796 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.797580 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.797848 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.797930 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.800886 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:07.800888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:07.800888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:07.800896 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:07.802537 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.802886 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.802887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.802887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.802947 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.803205 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.803207 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.803207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.803366 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:07.803455 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.803674 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.803755 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:25:07.830864 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:07.822611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46546 (local address 127.24.66.65:35857)
0812 05:25:07.822686 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:07.822688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:07.822845 (+ 157us) server_negotiation.cc:407] Connection header received
0812 05:25:07.822927 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:07.822928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:07.822940 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:07.823023 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:07.823701 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.824238 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.825192 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:07.825413 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:07.825533 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:07.828199 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:07.828201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:07.828201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:07.828210 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:07.829727 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.830101 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.830102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.830102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.830159 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:07.830400 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:07.830401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:07.830401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:07.830503 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:07.830581 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:07.830725 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 05:25:07.830792 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 05:25:07.831729 25547 tablet_service.cc:2908] Rejecting scan request for tablet f5511357fd3d49b8906991c9e942874b: Uninitialized: clean time has not yet been initialized
I20260812 05:25:07.832731 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 25504
I20260812 05:25:07.840040 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 25643
I20260812 05:25:07.846621 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 25783
I20260812 05:25:07.852954 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 25405
2026-08-12T05:25:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2402 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25964](info): set up 2 sockets
Aug 12 05:25:07 dist-test-slave-zpfg krb5kdc[25964](info): commencing operation
krb5kdc: starting...
W20260812 05:25:09.020562 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:09 dist-test-slave-zpfg krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:25:09.158262 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:46175
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:36305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:46175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:09.248145 25978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:09.248440 25978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:09.248481 25978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:09.250233 25978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:09.250316 25978 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 05:25:09.250358 25978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:09.250384 25978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:09.250408 25978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:09.250435 25978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:09.252226 25978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:36305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:46175
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:46175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:09.252516 25978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:09.252792 25978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:09.258785 25984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:09.258849 25978 server_base.cc:1034] running on GCE node
W20260812 05:25:09.258805 25986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:09.258798 25983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:09.259447 25978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:09.259835 25978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:09.261027 25978 hybrid_clock.cc:648] HybridClock initialized: now 1786512309261063 us; error 102 us; skew 500 ppm
Aug 12 05:25:09 dist-test-slave-zpfg krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:09.264345 25978 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:09.264820 25978 webserver.cc:460] Webserver started at http://127.24.66.126:44757/ using document root <none> and password file <none>
I20260812 05:25:09.265033 25978 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:09.265079 25978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:09.265184 25978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:09.266431 25978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "3af0f2a314a542b8b0f183755a78abc7"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "830a7867f5f3d917392434bf9e39704f"
server_key_iv: "76a302734eaee202932ed6af9a831d5e"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.266822 25978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "3af0f2a314a542b8b0f183755a78abc7"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "830a7867f5f3d917392434bf9e39704f"
server_key_iv: "76a302734eaee202932ed6af9a831d5e"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.268325 25978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:25:09.269272 25993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:09.269508 25978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:09.269621 25978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "3af0f2a314a542b8b0f183755a78abc7"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "830a7867f5f3d917392434bf9e39704f"
server_key_iv: "76a302734eaee202932ed6af9a831d5e"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.269722 25978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:09.302289 25978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:09.304432 25978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:09.304622 25978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:09.310055 25978 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:46175
I20260812 05:25:09.310113 26057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:46175 every 8 connection(s)
I20260812 05:25:09.310470 25978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:09.311216 26058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:09.313613 26058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7: Bootstrap starting.
I20260812 05:25:09.314270 26058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:09.314610 26058 log.cc:824] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:09.314908 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 25978
I20260812 05:25:09.314996 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:09.315176 24841 external_mini_cluster.cc:1428] Setting key a920524ddfd9f33d130e1e95b4135a65
I20260812 05:25:09.315532 26058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7: No bootstrap required, opened a new log
I20260812 05:25:09.317871 26058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af0f2a314a542b8b0f183755a78abc7" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46175 } }
I20260812 05:25:09.318055 26058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:09.318096 26058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3af0f2a314a542b8b0f183755a78abc7, State: Initialized, Role: FOLLOWER
I20260812 05:25:09.318243 26058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af0f2a314a542b8b0f183755a78abc7" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46175 } }
I20260812 05:25:09.318338 26058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:09.318431 26058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:09.318491 26058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:09.319231 26058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af0f2a314a542b8b0f183755a78abc7" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46175 } }
I20260812 05:25:09.319329 26058 leader_election.cc:302] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3af0f2a314a542b8b0f183755a78abc7; no voters:
I20260812 05:25:09.319577 26058 leader_election.cc:288] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:09.319612 26063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:09.319720 26063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [term 1 LEADER]: Becoming Leader. State: Replica: 3af0f2a314a542b8b0f183755a78abc7, State: Running, Role: LEADER
I20260812 05:25:09.319816 26063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af0f2a314a542b8b0f183755a78abc7" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46175 } }
I20260812 05:25:09.319881 26058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:09.320165 26063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3af0f2a314a542b8b0f183755a78abc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af0f2a314a542b8b0f183755a78abc7" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46175 } } }
I20260812 05:25:09.320238 26063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:09.320374 26063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3af0f2a314a542b8b0f183755a78abc7. Latest consensus state: current_term: 1 leader_uuid: "3af0f2a314a542b8b0f183755a78abc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3af0f2a314a542b8b0f183755a78abc7" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46175 } } }
I20260812 05:25:09.320449 26063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:09.320746 26069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:09.321277 26069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:09.322999 26069 catalog_manager.cc:1347] Generated new cluster ID: 2171b5df10a14218bc6dd89df1ba7459
I20260812 05:25:09.323058 26069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:25:09 dist-test-slave-zpfg krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:09.332887 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.315724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57280 (local address 127.24.66.126:46175)
0812 05:25:09.315913 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:09.315922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:09.315959 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:25:09.317849 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:09.317858 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:09.317916 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:09.318045 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:09.318931 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.320345 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.321616 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.322216 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.322355 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.328895 (+ 6540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:09.328900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:09.328901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:09.328916 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:09.331082 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.331547 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.331550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.331550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.331673 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.331896 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.331897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.331897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.332214 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:09.332359 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:09.332487 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 05:25:09.332632 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
I20260812 05:25:09.345888 26069 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:09.346580 26069 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:09.351897 26069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7: Generated new TSK 0
I20260812 05:25:09.352104 26069 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:25:09.381153 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:46175
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:36305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:09.476794 26087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:09.477123 26087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:09.477155 26087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:09.478595 26087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:09.478643 26087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:09.478675 26087 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:25:09.478696 26087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:09.480363 26087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:36305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.66.126:46175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:09.480598 26087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:09.480851 26087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:09.486590 26096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:09.486579 26093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:09.486722 26094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:09.486860 26087 server_base.cc:1034] running on GCE node
I20260812 05:25:09.487128 26087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:09.487479 26087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:09.488675 26087 hybrid_clock.cc:648] HybridClock initialized: now 1786512309488654 us; error 41 us; skew 500 ppm
Aug 12 05:25:09 dist-test-slave-zpfg krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:09.491629 26087 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:25:09.492111 26087 webserver.cc:460] Webserver started at http://127.24.66.65:35559/ using document root <none> and password file <none>
I20260812 05:25:09.492378 26087 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:09.492450 26087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:09.492602 26087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:09.493728 26087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "6dad9c8f26c945e282f66b67186e25ac"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "69a9448cac196d61e07d02929e57a9d6"
server_key_iv: "a074f07004263dbdd31cfcef14319ae8"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.494117 26087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "6dad9c8f26c945e282f66b67186e25ac"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "69a9448cac196d61e07d02929e57a9d6"
server_key_iv: "a074f07004263dbdd31cfcef14319ae8"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.495609 26087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:25:09.496492 26103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:09.496686 26087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:09.496797 26087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "6dad9c8f26c945e282f66b67186e25ac"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "69a9448cac196d61e07d02929e57a9d6"
server_key_iv: "a074f07004263dbdd31cfcef14319ae8"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.496891 26087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:09.515605 26087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:09.518033 26087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:09.518214 26087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:09.518786 26087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:09.518850 26087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:09.518895 26087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:09.518944 26087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:25:09 dist-test-slave-zpfg krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:09.527076 26087 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:36749
I20260812 05:25:09.527223 26223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:36749 every 8 connection(s)
I20260812 05:25:09.527565 26087 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:09.527930 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 26087
I20260812 05:25:09.528031 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:09.528204 24841 external_mini_cluster.cc:1428] Setting key 43836ea68633474bca5728b8b47d83fc
I20260812 05:25:09.531571 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.519642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:60569 (local address 127.24.66.126:46175)
0812 05:25:09.519721 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:09.519724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:09.520032 (+ 308us) server_negotiation.cc:407] Connection header received
0812 05:25:09.520949 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:09.520950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:09.520975 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:09.521115 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:09.522264 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.523000 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.524185 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.524452 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.524549 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.527457 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:09.527459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:09.527459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:09.527471 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:09.529746 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.530318 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.530319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.530319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.530410 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.530672 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.530673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.530673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.530844 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:09.530925 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:09.531420 (+ 495us) server_negotiation.cc:299] Negotiation successful
0812 05:25:09.531505 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:25:09.532003 26112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.519617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:46175 (local address 127.24.66.65:60569)
0812 05:25:09.519851 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:09.519908 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:09.520766 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:09.521258 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:09.521263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:09.521800 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:09.522037 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.522043 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.523111 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.523112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.524049 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.524052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.524582 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.524584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.524747 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.525444 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:09.525454 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:09.527333 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:09.529887 (+ 2554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.529894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.529896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.530233 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.530479 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.530480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.530480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.530601 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.530940 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:09.530949 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:09.531234 (+ 285us) client_negotiation.cc:769] Sending connection context
0812 05:25:09.531391 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 05:25:09.531685 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":120,"threads_started":1}
I20260812 05:25:09.532644 26224 heartbeater.cc:347] Connected to a master server at 127.24.66.126:46175
I20260812 05:25:09.532773 26224 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:09.532996 26224 heartbeater.cc:511] Master 127.24.66.126:46175 requested a full tablet report, sending...
I20260812 05:25:09.533579 26009 ts_manager.cc:194] Registered new tserver with Master: 6dad9c8f26c945e282f66b67186e25ac (127.24.66.65:36749)
I20260812 05:25:09.534670 26009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:60569
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
I20260812 05:25:09.540925 26231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.533874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:46175 (local address 127.24.66.65:52455)
0812 05:25:09.534481 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:09.534502 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:09.534571 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:09.534716 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:09.534717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:09.534954 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:09.535038 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.535041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.536080 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.536081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.536695 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.536697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.537211 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.537212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.537275 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.537831 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:09.537842 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:09.538239 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:09.539890 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.539891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.539892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.540147 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.540347 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.540348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.540348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.540423 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.540658 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:09.540658 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:09.540700 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 05:25:09.540752 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 05:25:09.540838 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":83,"threads_started":1}
I20260812 05:25:09.540937 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.534157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:52455 (local address 127.24.66.126:46175)
0812 05:25:09.534209 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:09.534211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:09.534576 (+ 365us) server_negotiation.cc:407] Connection header received
0812 05:25:09.534629 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:09.534630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:09.534644 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:09.534708 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:09.535129 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.535955 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.536797 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.537124 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.537204 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.538359 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:09.538361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:09.538362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:09.538370 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:09.539805 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.540224 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.540225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.540225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.540278 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.540487 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.540488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.540488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.540596 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:09.540651 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:09.540767 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 05:25:09.540876 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:25:09.561357 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:46175
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:36305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:09.649593 26234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:09.649829 26234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:09.649857 26234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:09.651257 26234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:09.651309 26234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:09.651331 26234 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:25:09.651352 26234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:09.652973 26234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:36305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.66.126:46175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:09.653258 26234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:09.653542 26234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:09.659628 26241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:09.659711 26243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:09.659629 26240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:09.659941 26234 server_base.cc:1034] running on GCE node
I20260812 05:25:09.660245 26234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:09.660524 26234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:09.661737 26234 hybrid_clock.cc:648] HybridClock initialized: now 1786512309661721 us; error 27 us; skew 500 ppm
Aug 12 05:25:09 dist-test-slave-zpfg krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:09.664486 26234 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:25:09.664994 26234 webserver.cc:460] Webserver started at http://127.24.66.66:44225/ using document root <none> and password file <none>
I20260812 05:25:09.665258 26234 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:09.665364 26234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:09.665508 26234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:09.666683 26234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "cb4bd9eda633479f8659f796c6547c53"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "a0e936da61b24538e49cd47dd60db9c7"
server_key_iv: "020fc2bb9a43157bce85f62ba16d178f"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.667133 26234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "cb4bd9eda633479f8659f796c6547c53"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "a0e936da61b24538e49cd47dd60db9c7"
server_key_iv: "020fc2bb9a43157bce85f62ba16d178f"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.668541 26234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:09.669304 26251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:09.669478 26234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:25:09.669579 26234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "cb4bd9eda633479f8659f796c6547c53"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "a0e936da61b24538e49cd47dd60db9c7"
server_key_iv: "020fc2bb9a43157bce85f62ba16d178f"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.669664 26234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:09.702005 26234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:09.704391 26234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:09.704588 26234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:09.705307 26234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:09.705370 26234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:09.705413 26234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:09.705428 26234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:25:09 dist-test-slave-zpfg krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:09.712955 26234 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:33929
I20260812 05:25:09.713085 26372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:33929 every 8 connection(s)
I20260812 05:25:09.713443 26234 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:09.715898 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.705832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:46737 (local address 127.24.66.126:46175)
0812 05:25:09.705912 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:09.705915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:09.706104 (+ 189us) server_negotiation.cc:407] Connection header received
0812 05:25:09.706679 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:09.706681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:09.706696 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:09.706785 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:09.707535 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.708402 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.709526 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.709931 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.710016 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.712830 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:09.712832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:09.712832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:09.712841 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:09.714528 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.714896 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.714897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.714897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.714949 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.715322 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.715323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.715323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.715428 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:09.715479 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:09.715763 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 05:25:09.715834 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 05:25:09.716328 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.705791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:46175 (local address 127.24.66.66:46737)
0812 05:25:09.705959 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:09.706001 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:09.706577 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:09.706873 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:09.706878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:09.707247 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:09.707453 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.707458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.708533 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.708534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.709413 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.709416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.710051 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.710052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.710187 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.710778 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:09.710786 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:09.712691 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:09.714599 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.714604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.714605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.714827 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.715044 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.715045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.715045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.715252 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.715503 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:09.715509 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:09.715631 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 05:25:09.715739 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 05:25:09.715863 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260812 05:25:09.716949 26373 heartbeater.cc:347] Connected to a master server at 127.24.66.126:46175
I20260812 05:25:09.717037 26373 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:09.717191 26373 heartbeater.cc:511] Master 127.24.66.126:46175 requested a full tablet report, sending...
I20260812 05:25:09.717572 26010 ts_manager.cc:194] Registered new tserver with Master: cb4bd9eda633479f8659f796c6547c53 (127.24.66.66:33929)
I20260812 05:25:09.717988 26010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:46737
I20260812 05:25:09.718245 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 26234
I20260812 05:25:09.718415 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:09.718637 24841 external_mini_cluster.cc:1428] Setting key 8ac31cf04b986f12ceb6fe57fc2793ed
I20260812 05:25:09.725632 26381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.718170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:46175 (local address 127.24.66.66:43603)
0812 05:25:09.718359 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:09.718369 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:09.718468 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:09.718651 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:09.718652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:09.718855 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:09.718935 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.718937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.719835 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.719836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.720969 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.720973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.721444 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.721445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.721525 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.722070 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:09.722075 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:09.722536 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:09.724341 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.724342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.724343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.724554 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.724809 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.724810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.724810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.724888 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.725338 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:09.725339 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:09.725385 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 05:25:09.725456 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 05:25:09.725547 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":69,"threads_started":1}
I20260812 05:25:09.725761 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.718220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:43603 (local address 127.24.66.126:46175)
0812 05:25:09.718261 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:09.718263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:09.718421 (+ 158us) server_negotiation.cc:407] Connection header received
0812 05:25:09.718555 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:09.718556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:09.718568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:09.718633 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:09.719002 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.719674 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.721068 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.721339 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.721428 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.722679 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:09.722680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:09.722681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:09.722688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:09.724224 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.724662 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.724663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.724664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.724731 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.724977 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.724979 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.724979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.725227 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:09.725320 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:09.725514 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 05:25:09.725611 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:25:09.750216 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:46175
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:36305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:09.836344 26384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:09.836624 26384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:09.836668 26384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:09.837958 26384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:09.838037 26384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:09.838074 26384 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:25:09.838115 26384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:09.839710 26384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:36305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.66.126:46175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:09.840019 26384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:09.840270 26384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:09.845883 26392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:09.845860 26389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:09.845983 26384 server_base.cc:1034] running on GCE node
W20260812 05:25:09.845867 26390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:09.846315 26384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:09.846647 26384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:09.847833 26384 hybrid_clock.cc:648] HybridClock initialized: now 1786512309847819 us; error 47 us; skew 500 ppm
Aug 12 05:25:09 dist-test-slave-zpfg krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:09.850639 26384 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:25:09.851146 26384 webserver.cc:460] Webserver started at http://127.24.66.67:45367/ using document root <none> and password file <none>
I20260812 05:25:09.851393 26384 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:09.851467 26384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:09.851641 26384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:09.852787 26384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "2fdd51a7c4cc409ebbed107e87774afb"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "bd3de4aaa42a3effcf48dd868c547253"
server_key_iv: "5aa242d419b95b40e6d77e10aacace28"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.853165 26384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "2fdd51a7c4cc409ebbed107e87774afb"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "bd3de4aaa42a3effcf48dd868c547253"
server_key_iv: "5aa242d419b95b40e6d77e10aacace28"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.854812 26384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:25:09.855649 26399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:09.855875 26384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:09.855978 26384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "2fdd51a7c4cc409ebbed107e87774afb"
format_stamp: "Formatted at 2026-08-12 05:25:09 on dist-test-slave-zpfg"
server_key: "bd3de4aaa42a3effcf48dd868c547253"
server_key_iv: "5aa242d419b95b40e6d77e10aacace28"
server_key_version: "encryptionkey@0"
I20260812 05:25:09.856062 26384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:09.868584 26384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:09.871194 26384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:09.871371 26384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:09.871932 26384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:09.872000 26384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:09.872047 26384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:09.872071 26384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:25:09 dist-test-slave-zpfg krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:09.880426 26384 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:40419
I20260812 05:25:09.880524 26517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:40419 every 8 connection(s)
I20260812 05:25:09.880916 26384 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:09.882886 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.872546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:44595 (local address 127.24.66.126:46175)
0812 05:25:09.872592 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:09.872594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:09.872925 (+ 331us) server_negotiation.cc:407] Connection header received
0812 05:25:09.873650 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:09.873652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:09.873668 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:09.873741 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:09.874807 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.875395 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.876596 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.876903 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.876975 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.879655 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:09.879657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:09.879657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:09.879665 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:09.881374 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.881784 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.881785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.881785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.881877 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.882070 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.882071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.882071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.882183 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:09.882233 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:09.882661 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 05:25:09.882730 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:25:09.882936 26408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.872534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:46175 (local address 127.24.66.67:44595)
0812 05:25:09.872710 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:09.872806 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:09.873522 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:09.873895 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:09.873901 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:09.874367 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:09.874688 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.874696 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.875507 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.875508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.876500 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.876503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.877021 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.877022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.877137 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.877748 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:09.877769 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:09.879542 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:09.881456 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.881460 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.881463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.881714 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.881922 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.881922 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.881922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.882010 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.882333 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:09.882390 (+ 57us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:09.882536 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 05:25:09.882627 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 05:25:09.882775 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260812 05:25:09.883446 26518 heartbeater.cc:347] Connected to a master server at 127.24.66.126:46175
I20260812 05:25:09.883538 26518 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:09.883733 26518 heartbeater.cc:511] Master 127.24.66.126:46175 requested a full tablet report, sending...
I20260812 05:25:09.884138 26009 ts_manager.cc:194] Registered new tserver with Master: 2fdd51a7c4cc409ebbed107e87774afb (127.24.66.67:40419)
I20260812 05:25:09.884615 26009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:44595
I20260812 05:25:09.886602 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 26384
I20260812 05:25:09.886739 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:09.886917 24841 external_mini_cluster.cc:1428] Setting key 9717ce808e0014d5e562f7aca67e5879
I20260812 05:25:09.888065 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:25:09.891999 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.884966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:47591 (local address 127.24.66.126:46175)
0812 05:25:09.885032 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:09.885034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:09.885043 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:09.885205 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:09.885206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:09.885216 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:09.885270 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:09.885696 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.886163 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.887479 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.887769 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.887856 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.888913 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:09.888915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:09.888916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:09.888924 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:09.890684 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.891128 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.891129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.891129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.891194 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.891463 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.891464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.891464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.891599 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:09.891672 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:09.891856 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 05:25:09.891930 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:25:09.892110 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.884733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:46175 (local address 127.24.66.67:47591)
0812 05:25:09.884964 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:09.884975 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:09.885046 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:09.885294 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:09.885295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:09.885525 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:09.885596 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.885598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.886508 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.886510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.887362 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.887365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.887881 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.887882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.887967 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.888502 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:09.888507 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:09.888818 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:09.890778 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.890779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.890780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.891035 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.891273 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:09.891274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:09.891275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:09.891351 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:09.891680 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:09.891681 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:09.891738 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 05:25:09.891819 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 05:25:09.891952 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":158,"threads_started":1}
Aug 12 05:25:09 dist-test-slave-zpfg krb5kdc[25964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 05:25:09 dist-test-slave-zpfg krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:09.915441 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.907530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57288 (local address 127.24.66.126:46175)
0812 05:25:09.907595 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:09.907597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:09.907613 (+ 16us) server_negotiation.cc:407] Connection header received
0812 05:25:09.907640 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:09.907641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:09.907650 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:09.907710 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:09.908330 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.908846 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.909724 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.909921 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.909984 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.912573 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:09.912575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:09.912575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:09.912582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:09.914096 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.914641 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.914643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.914643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.914705 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:09.914906 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:09.914906 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:09.914907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:09.915021 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:09.915100 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:09.915274 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 05:25:09.915343 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:25:09.917505 26009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:09.919453 26009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:09.926713 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.922439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:33929 (local address 127.0.0.1:50096)
0812 05:25:09.922828 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:09.922838 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:09.922878 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:09.923327 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:09.923328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:09.923330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:09.923366 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.923367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.924684 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.924685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.925650 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.925653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.926384 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.926386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.926466 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.926479 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 05:25:09.926558 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 05:25:09.926614 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260812 05:25:09.926724 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.922661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50096 (local address 127.24.66.66:33929)
0812 05:25:09.923103 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:09.923107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:09.923117 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:09.923210 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:09.923213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:09.923232 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:09.923329 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:09.923432 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.924581 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.925745 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.926270 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.926410 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.926480 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:09.926573 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 05:25:09.926614 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":89,"threads_started":1}
I20260812 05:25:09.927008 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.922153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.24.66.67:40419)
0812 05:25:09.922398 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:09.922403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:09.922811 (+ 408us) server_negotiation.cc:407] Connection header received
0812 05:25:09.922866 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:09.922869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:09.922887 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:09.922969 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:09.923207 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.924104 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.925524 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.926310 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.926461 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.926526 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:09.926730 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 05:25:09.926782 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":115,"threads_started":1}
I20260812 05:25:09.927172 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.922188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49124 (local address 127.24.66.65:36749)
0812 05:25:09.922466 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:09.922468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:09.922850 (+ 382us) server_negotiation.cc:407] Connection header received
0812 05:25:09.922895 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:09.922897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:09.922915 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:09.922983 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:09.923240 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.924528 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.925780 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.926513 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.926575 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.926636 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:09.927023 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 05:25:09.927069 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":96,"threads_started":1}
I20260812 05:25:09.926930 26535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.922053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:36749 (local address 127.0.0.1:49124)
0812 05:25:09.922541 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:09.922588 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:09.922698 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:09.923059 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:09.923060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:09.923063 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:09.923133 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.923136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.924645 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.924646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.925714 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.925717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.926668 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.926669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.926743 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.926746 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:09.926791 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 05:25:09.926831 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":30,"thread_start_us":233,"threads_started":1}
I20260812 05:25:09.927546 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:09.922053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:40419 (local address 127.0.0.1:42258)
0812 05:25:09.922574 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:09.922606 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:09.922737 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:09.923027 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:09.923030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:09.923033 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:09.923122 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.923124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.924206 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.924207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.925424 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:09.925427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:09.926564 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:09.926565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:09.926630 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:09.926635 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:25:09.927419 (+ 784us) client_negotiation.cc:240] Negotiation successful
0812 05:25:09.927470 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":5,"thread_start_us":50,"threads_started":1}
I20260812 05:25:09.927968 26303 tablet_service.cc:1467] Processing CreateTablet for tablet a0bd22cceead442abf5895ccaeae1b70 (DEFAULT_TABLE table=test-table [id=b0235819db2b4bc4a07862e9140e2036]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:09.927963 26155 tablet_service.cc:1467] Processing CreateTablet for tablet a0bd22cceead442abf5895ccaeae1b70 (DEFAULT_TABLE table=test-table [id=b0235819db2b4bc4a07862e9140e2036]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:09.928226 26449 tablet_service.cc:1467] Processing CreateTablet for tablet a0bd22cceead442abf5895ccaeae1b70 (DEFAULT_TABLE table=test-table [id=b0235819db2b4bc4a07862e9140e2036]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:09.928392 26303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0bd22cceead442abf5895ccaeae1b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:09.928392 26155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0bd22cceead442abf5895ccaeae1b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:09.928536 26449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0bd22cceead442abf5895ccaeae1b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:09.931703 26541 tablet_bootstrap.cc:492] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb: Bootstrap starting.
I20260812 05:25:09.932792 26541 tablet_bootstrap.cc:654] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:09.933041 26543 tablet_bootstrap.cc:492] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: Bootstrap starting.
I20260812 05:25:09.933045 26542 tablet_bootstrap.cc:492] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: Bootstrap starting.
I20260812 05:25:09.933553 26541 log.cc:824] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:09.933938 26543 tablet_bootstrap.cc:654] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:09.934574 26543 log.cc:824] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:09.934778 26542 tablet_bootstrap.cc:654] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:09.935220 26541 tablet_bootstrap.cc:492] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb: No bootstrap required, opened a new log
I20260812 05:25:09.935343 26541 ts_tablet_manager.cc:1397] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 05:25:09.935461 26542 log.cc:824] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:09.935544 26543 tablet_bootstrap.cc:492] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: No bootstrap required, opened a new log
I20260812 05:25:09.935636 26543 ts_tablet_manager.cc:1397] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 05:25:09.936578 26542 tablet_bootstrap.cc:492] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: No bootstrap required, opened a new log
I20260812 05:25:09.936672 26542 ts_tablet_manager.cc:1397] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 05:25:09.938433 26543 raft_consensus.cc:348] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } }
I20260812 05:25:09.938617 26543 raft_consensus.cc:374] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:09.938665 26543 raft_consensus.cc:729] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb4bd9eda633479f8659f796c6547c53, State: Initialized, Role: FOLLOWER
I20260812 05:25:09.938807 26543 consensus_queue.cc:260] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } }
I20260812 05:25:09.939090 26541 raft_consensus.cc:348] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } }
I20260812 05:25:09.939239 26543 ts_tablet_manager.cc:1428] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260812 05:25:09.939283 26541 raft_consensus.cc:374] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:09.939337 26541 raft_consensus.cc:729] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdd51a7c4cc409ebbed107e87774afb, State: Initialized, Role: FOLLOWER
I20260812 05:25:09.939357 26373 heartbeater.cc:503] Master 127.24.66.126:46175 was elected leader, sending a full tablet report...
I20260812 05:25:09.939477 26541 consensus_queue.cc:260] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } }
I20260812 05:25:09.939889 26541 ts_tablet_manager.cc:1428] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260812 05:25:09.940042 26518 heartbeater.cc:503] Master 127.24.66.126:46175 was elected leader, sending a full tablet report...
I20260812 05:25:09.939944 26542 raft_consensus.cc:348] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } }
I20260812 05:25:09.940091 26542 raft_consensus.cc:374] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:09.940142 26542 raft_consensus.cc:729] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dad9c8f26c945e282f66b67186e25ac, State: Initialized, Role: FOLLOWER
I20260812 05:25:09.940277 26542 consensus_queue.cc:260] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } }
I20260812 05:25:09.940647 26542 ts_tablet_manager.cc:1428] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 05:25:09.940794 26224 heartbeater.cc:503] Master 127.24.66.126:46175 was elected leader, sending a full tablet report...
W20260812 05:25:09.964219 26375 tablet.cc:2369] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:25:10.028517 26225 tablet.cc:2369] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:25:10.058226 26548 raft_consensus.cc:482] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:10.058480 26548 raft_consensus.cc:504] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } }
I20260812 05:25:10.058972 26548 leader_election.cc:288] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dad9c8f26c945e282f66b67186e25ac (127.24.66.65:36749), cb4bd9eda633479f8659f796c6547c53 (127.24.66.66:33929)
I20260812 05:25:10.063289 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.059362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:57763 (local address 127.24.66.65:36749)
0812 05:25:10.059426 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:10.059429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:10.059443 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:25:10.059508 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:10.059509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:10.059520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:10.059579 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:10.059784 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.060577 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.062077 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.062770 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.062848 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.062860 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:10.063159 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 05:25:10.063221 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:25:10.063279 26408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.059315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:36749 (local address 127.24.66.67:57763)
0812 05:25:10.059365 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:10.059377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:10.059427 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:10.059585 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:10.059587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:10.059588 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:10.059673 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.059675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.060652 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.060653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.061976 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.061979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.062881 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.062883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.063055 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.063059 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:10.063135 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 05:25:10.063187 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 05:25:10.063321 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.059452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:33929 (local address 127.24.66.67:52411)
0812 05:25:10.059685 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:10.059696 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:10.059751 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:10.059879 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:10.059879 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:10.059881 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:10.059931 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.059932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.060835 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.060836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.062083 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.062085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.062898 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.062899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.063135 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.063139 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:10.063190 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 05:25:10.063218 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":161,"threads_started":1}
I20260812 05:25:10.063325 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.059603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:52411 (local address 127.24.66.66:33929)
0812 05:25:10.059657 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:10.059659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:10.059763 (+ 104us) server_negotiation.cc:407] Connection header received
0812 05:25:10.059796 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:10.059797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:10.059808 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:10.059865 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:10.060022 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.060764 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.062181 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.062828 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.062895 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.062902 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:10.063209 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 05:25:10.063257 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 05:25:10.063720 26176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bd22cceead442abf5895ccaeae1b70" candidate_uuid: "2fdd51a7c4cc409ebbed107e87774afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dad9c8f26c945e282f66b67186e25ac" is_pre_election: true
I20260812 05:25:10.063794 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bd22cceead442abf5895ccaeae1b70" candidate_uuid: "2fdd51a7c4cc409ebbed107e87774afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb4bd9eda633479f8659f796c6547c53" is_pre_election: true
I20260812 05:25:10.063920 26176 raft_consensus.cc:2454] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fdd51a7c4cc409ebbed107e87774afb in term 0.
I20260812 05:25:10.063949 26324 raft_consensus.cc:2454] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fdd51a7c4cc409ebbed107e87774afb in term 0.
I20260812 05:25:10.064193 26403 leader_election.cc:302] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdd51a7c4cc409ebbed107e87774afb, 6dad9c8f26c945e282f66b67186e25ac; no voters:
I20260812 05:25:10.064363 26548 raft_consensus.cc:2781] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:10.064433 26548 raft_consensus.cc:482] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:10.064451 26548 raft_consensus.cc:3037] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:10.065356 26548 raft_consensus.cc:504] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } }
I20260812 05:25:10.065498 26548 leader_election.cc:288] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [CANDIDATE]: Term 1 election: Requested vote from peers 6dad9c8f26c945e282f66b67186e25ac (127.24.66.65:36749), cb4bd9eda633479f8659f796c6547c53 (127.24.66.66:33929)
I20260812 05:25:10.065663 26176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bd22cceead442abf5895ccaeae1b70" candidate_uuid: "2fdd51a7c4cc409ebbed107e87774afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dad9c8f26c945e282f66b67186e25ac"
I20260812 05:25:10.065711 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0bd22cceead442abf5895ccaeae1b70" candidate_uuid: "2fdd51a7c4cc409ebbed107e87774afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb4bd9eda633479f8659f796c6547c53"
I20260812 05:25:10.065770 26176 raft_consensus.cc:3037] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:10.065788 26324 raft_consensus.cc:3037] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:10.066843 26324 raft_consensus.cc:2454] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fdd51a7c4cc409ebbed107e87774afb in term 1.
I20260812 05:25:10.066915 26176 raft_consensus.cc:2454] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fdd51a7c4cc409ebbed107e87774afb in term 1.
I20260812 05:25:10.067041 26403 leader_election.cc:302] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdd51a7c4cc409ebbed107e87774afb, cb4bd9eda633479f8659f796c6547c53; no voters:
I20260812 05:25:10.067147 26548 raft_consensus.cc:2781] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:10.067294 26548 raft_consensus.cc:686] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 1 LEADER]: Becoming Leader. State: Replica: 2fdd51a7c4cc409ebbed107e87774afb, State: Running, Role: LEADER
I20260812 05:25:10.067405 26548 consensus_queue.cc:237] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } }
I20260812 05:25:10.068405 26009 catalog_manager.cc:5495] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fdd51a7c4cc409ebbed107e87774afb (127.24.66.67). New cstate: current_term: 1 leader_uuid: "2fdd51a7c4cc409ebbed107e87774afb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } health_report { overall_health: UNKNOWN } } }
I20260812 05:25:10.111675 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.108662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42272 (local address 127.24.66.67:40419)
0812 05:25:10.108729 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:10.108731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:10.108739 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:10.108786 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:10.108787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:10.108806 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:10.108867 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:10.109119 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.109625 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.110524 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.110750 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.110820 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.111010 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:10.111109 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:10.111185 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:10.111228 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 05:25:10.111249 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:25:10.113848 26176 raft_consensus.cc:1261] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac [term 1 FOLLOWER]: Refusing update from remote peer 2fdd51a7c4cc409ebbed107e87774afb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:10.113853 26324 raft_consensus.cc:1261] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53 [term 1 FOLLOWER]: Refusing update from remote peer 2fdd51a7c4cc409ebbed107e87774afb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:10.114454 26548 consensus_queue.cc:1035] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:10.114660 26551 consensus_queue.cc:1035] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:10.118263 26556 mvcc.cc:205] Tried to move back new op lower bound from 7317554422222901248 to 7317554422036873216. Current Snapshot: MvccSnapshot[applied={T|T < 7317554422222901248}]
I20260812 05:25:10.118592 26557 mvcc.cc:205] Tried to move back new op lower bound from 7317554422222901248 to 7317554422036873216. Current Snapshot: MvccSnapshot[applied={T|T < 7317554422222901248}]
I20260812 05:25:10.122210 26010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:10.122395 26010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:10.123327 26010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7: Sending DeleteTablet for 3 replicas of tablet a0bd22cceead442abf5895ccaeae1b70
I20260812 05:25:10.123862 26449 tablet_service.cc:1514] Processing DeleteTablet for tablet a0bd22cceead442abf5895ccaeae1b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:10 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:42258
I20260812 05:25:10.123903 26155 tablet_service.cc:1514] Processing DeleteTablet for tablet a0bd22cceead442abf5895ccaeae1b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:10 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:49124
I20260812 05:25:10.124001 26010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:10.124148 26010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:10.124130 26303 tablet_service.cc:1514] Processing DeleteTablet for tablet a0bd22cceead442abf5895ccaeae1b70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:10 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:50096
I20260812 05:25:10.124655 26566 tablet_replica.cc:330] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb: stopping tablet replica
I20260812 05:25:10.124735 26567 tablet_replica.cc:330] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: stopping tablet replica
I20260812 05:25:10.124861 26566 raft_consensus.cc:2229] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:10.124887 26567 raft_consensus.cc:2229] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:10.125061 26567 raft_consensus.cc:2258] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:10.125057 26566 raft_consensus.cc:2258] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:10.125767 26449 tablet_service.cc:1467] Processing CreateTablet for tablet 77014d38e28345eda8f318ef76052807 (DEFAULT_TABLE table=test-table [id=b592f31e210849a7b3ff9e1bf338b7f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:10.125787 26155 tablet_service.cc:1467] Processing CreateTablet for tablet 77014d38e28345eda8f318ef76052807 (DEFAULT_TABLE table=test-table [id=b592f31e210849a7b3ff9e1bf338b7f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:10.125906 26449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77014d38e28345eda8f318ef76052807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:10.125906 26155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77014d38e28345eda8f318ef76052807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:10.126823 26567 ts_tablet_manager.cc:1905] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:10.127743 26303 tablet_service.cc:1467] Processing CreateTablet for tablet 77014d38e28345eda8f318ef76052807 (DEFAULT_TABLE table=test-table [id=b592f31e210849a7b3ff9e1bf338b7f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:10.127859 26303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77014d38e28345eda8f318ef76052807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:10.129027 26543 tablet_bootstrap.cc:492] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53: Bootstrap starting.
I20260812 05:25:10.129127 26542 tablet_bootstrap.cc:492] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac: Bootstrap starting.
I20260812 05:25:10.129878 26542 tablet_bootstrap.cc:654] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:10.129877 26543 tablet_bootstrap.cc:654] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:10.130411 26541 tablet_bootstrap.cc:492] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb: Bootstrap starting.
I20260812 05:25:10.130594 26567 ts_tablet_manager.cc:1918] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:10.130667 26567 log.cc:1196] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/a0bd22cceead442abf5895ccaeae1b70
I20260812 05:25:10.130944 26567 ts_tablet_manager.cc:1939] T a0bd22cceead442abf5895ccaeae1b70 P cb4bd9eda633479f8659f796c6547c53: Deleting consensus metadata
I20260812 05:25:10.131074 26541 tablet_bootstrap.cc:654] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:10.131991 25997 catalog_manager.cc:4841] TS cb4bd9eda633479f8659f796c6547c53 (127.24.66.66:33929): tablet a0bd22cceead442abf5895ccaeae1b70 (table test-table [id=b0235819db2b4bc4a07862e9140e2036]) successfully deleted
I20260812 05:25:10.132095 26541 tablet_bootstrap.cc:492] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb: No bootstrap required, opened a new log
I20260812 05:25:10.132153 26541 ts_tablet_manager.cc:1397] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:25:10.132298 26541 raft_consensus.cc:348] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.132390 26541 raft_consensus.cc:374] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:10.132414 26541 raft_consensus.cc:729] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdd51a7c4cc409ebbed107e87774afb, State: Initialized, Role: FOLLOWER
I20260812 05:25:10.132491 26541 consensus_queue.cc:260] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.132586 26541 ts_tablet_manager.cc:1428] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:10.132659 26566 ts_tablet_manager.cc:1905] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:10.134511 26565 tablet_replica.cc:330] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: stopping tablet replica
I20260812 05:25:10.134663 26565 raft_consensus.cc:2229] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:10.134811 26565 raft_consensus.cc:2258] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:10.135254 26542 tablet_bootstrap.cc:492] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac: No bootstrap required, opened a new log
I20260812 05:25:10.135329 26542 ts_tablet_manager.cc:1397] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 05:25:10.135888 26542 raft_consensus.cc:348] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.135946 26543 tablet_bootstrap.cc:492] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53: No bootstrap required, opened a new log
I20260812 05:25:10.135977 26542 raft_consensus.cc:374] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:10.136000 26542 raft_consensus.cc:729] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dad9c8f26c945e282f66b67186e25ac, State: Initialized, Role: FOLLOWER
I20260812 05:25:10.135993 26543 ts_tablet_manager.cc:1397] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260812 05:25:10.136052 26542 consensus_queue.cc:260] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.136341 26542 ts_tablet_manager.cc:1428] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:10.136389 26543 raft_consensus.cc:348] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.136466 26543 raft_consensus.cc:374] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:10.136494 26543 raft_consensus.cc:729] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb4bd9eda633479f8659f796c6547c53, State: Initialized, Role: FOLLOWER
I20260812 05:25:10.136547 26543 consensus_queue.cc:260] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.136657 26543 ts_tablet_manager.cc:1428] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:10.137104 26565 ts_tablet_manager.cc:1905] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:10.137746 26566 ts_tablet_manager.cc:1918] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:10.137915 26566 log.cc:1196] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/a0bd22cceead442abf5895ccaeae1b70
I20260812 05:25:10.138388 26566 ts_tablet_manager.cc:1939] T a0bd22cceead442abf5895ccaeae1b70 P 2fdd51a7c4cc409ebbed107e87774afb: Deleting consensus metadata
I20260812 05:25:10.138980 25995 catalog_manager.cc:4841] TS 2fdd51a7c4cc409ebbed107e87774afb (127.24.66.67:40419): tablet a0bd22cceead442abf5895ccaeae1b70 (table test-table [id=b0235819db2b4bc4a07862e9140e2036]) successfully deleted
I20260812 05:25:10.139726 26565 ts_tablet_manager.cc:1918] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:10.139806 26565 log.cc:1196] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/wal/wals/a0bd22cceead442abf5895ccaeae1b70
I20260812 05:25:10.140058 26565 ts_tablet_manager.cc:1939] T a0bd22cceead442abf5895ccaeae1b70 P 6dad9c8f26c945e282f66b67186e25ac: Deleting consensus metadata
I20260812 05:25:10.140558 25997 catalog_manager.cc:4841] TS 6dad9c8f26c945e282f66b67186e25ac (127.24.66.65:36749): tablet a0bd22cceead442abf5895ccaeae1b70 (table test-table [id=b0235819db2b4bc4a07862e9140e2036]) successfully deleted
I20260812 05:25:10.167049 26547 raft_consensus.cc:482] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:10.167273 26547 raft_consensus.cc:504] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.167730 26547 leader_election.cc:288] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dad9c8f26c945e282f66b67186e25ac (127.24.66.65:36749), 2fdd51a7c4cc409ebbed107e87774afb (127.24.66.67:40419)
I20260812 05:25:10.171211 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.168090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:38099 (local address 127.24.66.67:40419)
0812 05:25:10.168137 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:10.168139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:10.168257 (+ 118us) server_negotiation.cc:407] Connection header received
0812 05:25:10.168369 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:10.168371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:10.168382 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:10.168438 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:10.168598 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.169066 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.170293 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.170804 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.170879 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.170888 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:10.171095 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 05:25:10.171138 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:25:10.171240 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.167920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:40419 (local address 127.24.66.66:38099)
0812 05:25:10.168195 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:10.168204 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:10.168282 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:10.168446 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:10.168447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:10.168448 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:10.168521 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.168522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.169168 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.169169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.170184 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.170187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.170933 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.170934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.171013 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.171018 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:25:10.171081 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 05:25:10.171127 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":67,"threads_started":1}
I20260812 05:25:10.171705 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.167873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:36749 (local address 127.24.66.66:39473)
0812 05:25:10.168000 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:10.168015 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:10.168066 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:10.168264 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:10.168265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:10.168266 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:10.168348 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.168350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.169329 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.169330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.170597 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.170600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.171451 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.171452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.171544 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.171548 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:10.171605 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:25:10.171643 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 05:25:10.171763 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.168038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:39473 (local address 127.24.66.65:36749)
0812 05:25:10.168156 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:10.168159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:10.168167 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:10.168196 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:10.168197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:10.168207 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:10.168258 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:10.168452 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.169220 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.170696 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.171194 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.171265 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.171274 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:10.171635 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 05:25:10.171682 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 05:25:10.171756 26471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77014d38e28345eda8f318ef76052807" candidate_uuid: "cb4bd9eda633479f8659f796c6547c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdd51a7c4cc409ebbed107e87774afb" is_pre_election: true
I20260812 05:25:10.171950 26471 raft_consensus.cc:2454] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb4bd9eda633479f8659f796c6547c53 in term 0.
I20260812 05:25:10.171995 26176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77014d38e28345eda8f318ef76052807" candidate_uuid: "cb4bd9eda633479f8659f796c6547c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dad9c8f26c945e282f66b67186e25ac" is_pre_election: true
I20260812 05:25:10.172117 26176 raft_consensus.cc:2454] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb4bd9eda633479f8659f796c6547c53 in term 0.
I20260812 05:25:10.172214 26253 leader_election.cc:302] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdd51a7c4cc409ebbed107e87774afb, cb4bd9eda633479f8659f796c6547c53; no voters:
I20260812 05:25:10.172453 26547 raft_consensus.cc:2781] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:10.172502 26547 raft_consensus.cc:482] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:10.172535 26547 raft_consensus.cc:3037] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:10.173548 26547 raft_consensus.cc:504] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.173710 26547 leader_election.cc:288] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [CANDIDATE]: Term 1 election: Requested vote from peers 6dad9c8f26c945e282f66b67186e25ac (127.24.66.65:36749), 2fdd51a7c4cc409ebbed107e87774afb (127.24.66.67:40419)
I20260812 05:25:10.173877 26176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77014d38e28345eda8f318ef76052807" candidate_uuid: "cb4bd9eda633479f8659f796c6547c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dad9c8f26c945e282f66b67186e25ac"
I20260812 05:25:10.173965 26176 raft_consensus.cc:3037] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:10.173919 26471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77014d38e28345eda8f318ef76052807" candidate_uuid: "cb4bd9eda633479f8659f796c6547c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdd51a7c4cc409ebbed107e87774afb"
I20260812 05:25:10.174016 26471 raft_consensus.cc:3037] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:10.174999 26471 raft_consensus.cc:2454] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb4bd9eda633479f8659f796c6547c53 in term 1.
I20260812 05:25:10.175089 26176 raft_consensus.cc:2454] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb4bd9eda633479f8659f796c6547c53 in term 1.
I20260812 05:25:10.175237 26253 leader_election.cc:302] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdd51a7c4cc409ebbed107e87774afb, cb4bd9eda633479f8659f796c6547c53; no voters:
I20260812 05:25:10.175350 26547 raft_consensus.cc:2781] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:10.175462 26547 raft_consensus.cc:686] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 1 LEADER]: Becoming Leader. State: Replica: cb4bd9eda633479f8659f796c6547c53, State: Running, Role: LEADER
I20260812 05:25:10.175540 26547 consensus_queue.cc:237] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.176455 26010 catalog_manager.cc:5495] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb4bd9eda633479f8659f796c6547c53 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "cb4bd9eda633479f8659f796c6547c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } health_report { overall_health: UNKNOWN } } }
Aug 12 05:25:10 dist-test-slave-zpfg krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:10.208256 26579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.199408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:46175 (local address 127.0.0.1:57300)
0812 05:25:10.199675 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:10.199689 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:10.199775 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:10.199949 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:10.199950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:10.200393 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:10.200484 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.200487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.201196 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.201197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.202034 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.202037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.202485 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.202486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.202549 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.203118 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:10.203123 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:10.204790 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:10.206856 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:10.206868 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:10.206870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:10.207137 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:10.207515 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:10.207516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:10.207516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:10.207584 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:10.207932 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:10.207940 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:10.207987 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 05:25:10.208060 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 05:25:10.208179 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":141,"threads_started":1}
I20260812 05:25:10.208381 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.199448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57300 (local address 127.24.66.126:46175)
0812 05:25:10.199538 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:10.199540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:10.199764 (+ 224us) server_negotiation.cc:407] Connection header received
0812 05:25:10.199853 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:10.199854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:10.199864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:10.199918 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:10.200574 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.201104 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.202130 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.202400 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.202466 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.204884 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:10.204885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:10.204886 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:10.204894 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:10.206704 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:10.207342 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:10.207343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:10.207343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:10.207413 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:10.207671 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:10.207672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:10.207673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:10.207837 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:10.207918 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:10.208124 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 05:25:10.208280 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":47}
I20260812 05:25:10.211390 26010 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:57300:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 05:25:10.211796 26010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:10.213706 26303 tablet_service.cc:1467] Processing CreateTablet for tablet 0499f9e90a5843bc8a45c15e26339265 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd98755570204b7b906b46d1a886849d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:25:10.213774 26449 tablet_service.cc:1467] Processing CreateTablet for tablet 0499f9e90a5843bc8a45c15e26339265 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd98755570204b7b906b46d1a886849d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:25:10.213801 26155 tablet_service.cc:1467] Processing CreateTablet for tablet 0499f9e90a5843bc8a45c15e26339265 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd98755570204b7b906b46d1a886849d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:25:10.213958 26449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0499f9e90a5843bc8a45c15e26339265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:10.213958 26155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0499f9e90a5843bc8a45c15e26339265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:10.213959 26303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0499f9e90a5843bc8a45c15e26339265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:10.216023 26543 tablet_bootstrap.cc:492] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: Bootstrap starting.
I20260812 05:25:10.216176 26542 tablet_bootstrap.cc:492] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac: Bootstrap starting.
I20260812 05:25:10.216650 26543 tablet_bootstrap.cc:654] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:10.216928 26542 tablet_bootstrap.cc:654] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:10.217430 26541 tablet_bootstrap.cc:492] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb: Bootstrap starting.
I20260812 05:25:10.218215 26541 tablet_bootstrap.cc:654] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:10.218329 26542 tablet_bootstrap.cc:492] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac: No bootstrap required, opened a new log
I20260812 05:25:10.218423 26542 ts_tablet_manager.cc:1397] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:25:10.218746 26542 raft_consensus.cc:348] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.218859 26542 raft_consensus.cc:374] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:10.218916 26542 raft_consensus.cc:729] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dad9c8f26c945e282f66b67186e25ac, State: Initialized, Role: FOLLOWER
I20260812 05:25:10.218923 26543 tablet_bootstrap.cc:492] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: No bootstrap required, opened a new log
I20260812 05:25:10.218976 26543 ts_tablet_manager.cc:1397] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 05:25:10.218992 26542 consensus_queue.cc:260] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.219156 26542 ts_tablet_manager.cc:1428] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:10.219400 26543 raft_consensus.cc:348] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.219414 26549 tablet_replica.cc:439] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } }
I20260812 05:25:10.219491 26543 raft_consensus.cc:374] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:10.219517 26543 raft_consensus.cc:729] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb4bd9eda633479f8659f796c6547c53, State: Initialized, Role: FOLLOWER
I20260812 05:25:10.219494 26549 tablet_replica.cc:442] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:10.219568 26543 consensus_queue.cc:260] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.219681 26543 ts_tablet_manager.cc:1428] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 05:25:10.219702 26541 tablet_bootstrap.cc:492] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb: No bootstrap required, opened a new log
I20260812 05:25:10.219748 26541 ts_tablet_manager.cc:1397] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:25:10.219869 26547 tablet_replica.cc:439] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } }
I20260812 05:25:10.219947 26547 tablet_replica.cc:442] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:10.220146 26541 raft_consensus.cc:348] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.220233 26541 raft_consensus.cc:374] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:10.220256 26541 raft_consensus.cc:729] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdd51a7c4cc409ebbed107e87774afb, State: Initialized, Role: FOLLOWER
I20260812 05:25:10.220300 26541 consensus_queue.cc:260] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.220389 26541 ts_tablet_manager.cc:1428] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:10.220641 26561 tablet_replica.cc:439] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } }
I20260812 05:25:10.220731 26561 tablet_replica.cc:442] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:10.233024 26547 raft_consensus.cc:482] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:10.233254 26547 raft_consensus.cc:504] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.233445 26547 leader_election.cc:288] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dad9c8f26c945e282f66b67186e25ac (127.24.66.65:36749), 2fdd51a7c4cc409ebbed107e87774afb (127.24.66.67:40419)
I20260812 05:25:10.233686 26471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0499f9e90a5843bc8a45c15e26339265" candidate_uuid: "cb4bd9eda633479f8659f796c6547c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdd51a7c4cc409ebbed107e87774afb" is_pre_election: true
I20260812 05:25:10.233655 26176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0499f9e90a5843bc8a45c15e26339265" candidate_uuid: "cb4bd9eda633479f8659f796c6547c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dad9c8f26c945e282f66b67186e25ac" is_pre_election: true
I20260812 05:25:10.233780 26176 raft_consensus.cc:2454] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb4bd9eda633479f8659f796c6547c53 in term 0.
I20260812 05:25:10.233778 26471 raft_consensus.cc:2454] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb4bd9eda633479f8659f796c6547c53 in term 0.
I20260812 05:25:10.233945 26253 leader_election.cc:302] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdd51a7c4cc409ebbed107e87774afb, cb4bd9eda633479f8659f796c6547c53; no voters:
I20260812 05:25:10.234166 26547 raft_consensus.cc:2781] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:10.234216 26547 raft_consensus.cc:482] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:10.234244 26547 raft_consensus.cc:3037] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:10.235137 26547 raft_consensus.cc:504] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.235333 26547 leader_election.cc:288] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [CANDIDATE]: Term 1 election: Requested vote from peers 6dad9c8f26c945e282f66b67186e25ac (127.24.66.65:36749), 2fdd51a7c4cc409ebbed107e87774afb (127.24.66.67:40419)
I20260812 05:25:10.235522 26471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0499f9e90a5843bc8a45c15e26339265" candidate_uuid: "cb4bd9eda633479f8659f796c6547c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdd51a7c4cc409ebbed107e87774afb"
I20260812 05:25:10.235639 26471 raft_consensus.cc:3037] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:10.235598 26176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0499f9e90a5843bc8a45c15e26339265" candidate_uuid: "cb4bd9eda633479f8659f796c6547c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dad9c8f26c945e282f66b67186e25ac"
I20260812 05:25:10.235692 26176 raft_consensus.cc:3037] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:10.236464 26471 raft_consensus.cc:2454] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb4bd9eda633479f8659f796c6547c53 in term 1.
I20260812 05:25:10.236609 26176 raft_consensus.cc:2454] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb4bd9eda633479f8659f796c6547c53 in term 1.
I20260812 05:25:10.236711 26253 leader_election.cc:302] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdd51a7c4cc409ebbed107e87774afb, cb4bd9eda633479f8659f796c6547c53; no voters:
I20260812 05:25:10.236799 26547 raft_consensus.cc:2781] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:10.236842 26547 raft_consensus.cc:686] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [term 1 LEADER]: Becoming Leader. State: Replica: cb4bd9eda633479f8659f796c6547c53, State: Running, Role: LEADER
I20260812 05:25:10.236891 26547 consensus_queue.cc:237] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } }
I20260812 05:25:10.237252 26572 tablet_replica.cc:439] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: TxnStatusTablet state changed. Reason: New leader cb4bd9eda633479f8659f796c6547c53. Latest consensus state: current_term: 1 leader_uuid: "cb4bd9eda633479f8659f796c6547c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } }
I20260812 05:25:10.237354 26572 tablet_replica.cc:442] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:25:10.237529 26585 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 05:25:10.237673 26585 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 05:25:10.237818 26010 catalog_manager.cc:5495] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb4bd9eda633479f8659f796c6547c53 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "cb4bd9eda633479f8659f796c6547c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260812 05:25:10.246155 26579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.242111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:33929 (local address 127.0.0.1:50100)
0812 05:25:10.242230 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:10.242245 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:10.242303 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:10.242586 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:10.242587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:10.242588 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:25:10.242651 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.242654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.243855 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.243857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.244884 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.244887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.245502 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.245504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.245577 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.245599 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:25:10.245932 (+ 333us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:25:10.245945 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 05:25:10.246002 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:25:10.246071 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 05:25:10.246158 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.242246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50100 (local address 127.24.66.66:33929)
0812 05:25:10.242409 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:10.242411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:10.242421 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:10.242466 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:10.242467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:10.242488 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:10.242562 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:10.242742 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.243739 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.244995 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.245411 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.245490 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.245714 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:10.245849 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:10.245931 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:10.246004 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 05:25:10.246045 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260812 05:25:10.248247 26471 raft_consensus.cc:1261] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb [term 1 FOLLOWER]: Refusing update from remote peer cb4bd9eda633479f8659f796c6547c53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:10.248255 26176 raft_consensus.cc:1261] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac [term 1 FOLLOWER]: Refusing update from remote peer cb4bd9eda633479f8659f796c6547c53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:10.248571 26572 consensus_queue.cc:1035] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:10.248791 26547 consensus_queue.cc:1035] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:10.249788 26549 tablet_replica.cc:439] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac: TxnStatusTablet state changed. Reason: New leader cb4bd9eda633479f8659f796c6547c53. Latest consensus state: current_term: 1 leader_uuid: "cb4bd9eda633479f8659f796c6547c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } }
I20260812 05:25:10.249902 26549 tablet_replica.cc:442] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:10.250473 26561 tablet_replica.cc:439] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb: TxnStatusTablet state changed. Reason: New leader cb4bd9eda633479f8659f796c6547c53. Latest consensus state: current_term: 1 leader_uuid: "cb4bd9eda633479f8659f796c6547c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } }
I20260812 05:25:10.250577 26561 tablet_replica.cc:442] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:10.252352 26572 tablet_replica.cc:439] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cb4bd9eda633479f8659f796c6547c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } }
I20260812 05:25:10.252442 26572 tablet_replica.cc:442] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:25:10.253548 26549 tablet_replica.cc:439] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb4bd9eda633479f8659f796c6547c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } }
I20260812 05:25:10.253706 26549 tablet_replica.cc:442] T 0499f9e90a5843bc8a45c15e26339265 P 6dad9c8f26c945e282f66b67186e25ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:10.255080 26547 tablet_replica.cc:439] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cb4bd9eda633479f8659f796c6547c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } }
I20260812 05:25:10.255177 26547 tablet_replica.cc:442] T 0499f9e90a5843bc8a45c15e26339265 P cb4bd9eda633479f8659f796c6547c53: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:25:10.255695 26555 mvcc.cc:205] Tried to move back new op lower bound from 7317554422773686272 to 7317554422730649600. Current Snapshot: MvccSnapshot[applied={T|T < 7317554422773686272}]
I20260812 05:25:10.256032 26561 tablet_replica.cc:439] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb4bd9eda633479f8659f796c6547c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 } } peers { permanent_uuid: "cb4bd9eda633479f8659f796c6547c53" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 33929 } } peers { permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 } } }
I20260812 05:25:10.256138 26561 tablet_replica.cc:442] T 0499f9e90a5843bc8a45c15e26339265 P 2fdd51a7c4cc409ebbed107e87774afb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:10.258432 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.255120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50112 (local address 127.24.66.66:33929)
0812 05:25:10.255189 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:10.255191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:10.255198 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:25:10.255220 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:10.255220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:10.255232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:10.255286 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:10.255512 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.256303 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.257392 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.257852 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.257977 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.258141 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:10.258181 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:10.258239 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:10.258307 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 05:25:10.258331 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:25:10.265846 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.262007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:35281 (local address 127.24.66.66:33929)
0812 05:25:10.262043 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:10.262045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:10.262125 (+ 80us) server_negotiation.cc:407] Connection header received
0812 05:25:10.262194 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:10.262195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:10.262206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:10.262268 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:10.262586 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.263297 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.264803 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.265123 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.265206 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.265464 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:10.265500 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:10.265580 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:10.265693 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 05:25:10.265716 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 05:25:10.266000 26593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.261743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:33929 (local address 127.24.66.66:35281)
0812 05:25:10.262052 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:10.262066 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:10.262141 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:10.262285 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:10.262285 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:10.262373 (+ 88us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:25:10.262450 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.262452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.263393 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.263394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.264687 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:10.264691 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.265243 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.265244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:10.265313 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.265336 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:25:10.265618 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:25:10.265621 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:10.265676 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 05:25:10.265768 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":173,"threads_started":1}
I20260812 05:25:10.269212 26471 raft_consensus.cc:1261] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb [term 1 FOLLOWER]: Refusing update from remote peer cb4bd9eda633479f8659f796c6547c53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:10.269240 26176 raft_consensus.cc:1261] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac [term 1 FOLLOWER]: Refusing update from remote peer cb4bd9eda633479f8659f796c6547c53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:10.269423 26547 consensus_queue.cc:1035] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fdd51a7c4cc409ebbed107e87774afb" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:10.270785 26572 consensus_queue.cc:1035] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dad9c8f26c945e282f66b67186e25ac" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 36749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:10.295347 26010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:10.295477 26010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:10.296366 26010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3af0f2a314a542b8b0f183755a78abc7: Sending DeleteTablet for 3 replicas of tablet 77014d38e28345eda8f318ef76052807
I20260812 05:25:10.296919 26155 tablet_service.cc:1514] Processing DeleteTablet for tablet 77014d38e28345eda8f318ef76052807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:10 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:49124
I20260812 05:25:10.297061 26565 tablet_replica.cc:330] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac: stopping tablet replica
I20260812 05:25:10.297120 26565 raft_consensus.cc:2229] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:10.297181 26565 raft_consensus.cc:2258] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:10.297276 26449 tablet_service.cc:1514] Processing DeleteTablet for tablet 77014d38e28345eda8f318ef76052807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:10 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:42258
I20260812 05:25:10.297461 26566 tablet_replica.cc:330] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb: stopping tablet replica
I20260812 05:25:10.297477 26303 tablet_service.cc:1514] Processing DeleteTablet for tablet 77014d38e28345eda8f318ef76052807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:10 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:50096
I20260812 05:25:10.297534 26566 raft_consensus.cc:2229] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:10.297581 26566 raft_consensus.cc:2258] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:10.297600 26567 tablet_replica.cc:330] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53: stopping tablet replica
I20260812 05:25:10.297644 26567 raft_consensus.cc:2229] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:10.297772 26567 raft_consensus.cc:2258] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:10.298306 26567 ts_tablet_manager.cc:1905] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:10.298570 26565 ts_tablet_manager.cc:1905] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:10.300736 26565 ts_tablet_manager.cc:1918] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:10.300802 26565 log.cc:1196] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-0/wal/wals/77014d38e28345eda8f318ef76052807
I20260812 05:25:10.300884 26567 ts_tablet_manager.cc:1918] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:10.300993 26567 log.cc:1196] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/77014d38e28345eda8f318ef76052807
I20260812 05:25:10.301059 26565 ts_tablet_manager.cc:1939] T 77014d38e28345eda8f318ef76052807 P 6dad9c8f26c945e282f66b67186e25ac: Deleting consensus metadata
I20260812 05:25:10.301275 26567 ts_tablet_manager.cc:1939] T 77014d38e28345eda8f318ef76052807 P cb4bd9eda633479f8659f796c6547c53: Deleting consensus metadata
I20260812 05:25:10.301597 25997 catalog_manager.cc:4841] TS 6dad9c8f26c945e282f66b67186e25ac (127.24.66.65:36749): tablet 77014d38e28345eda8f318ef76052807 (table test-table [id=b592f31e210849a7b3ff9e1bf338b7f5]) successfully deleted
I20260812 05:25:10.301882 25997 catalog_manager.cc:4841] TS cb4bd9eda633479f8659f796c6547c53 (127.24.66.66:33929): tablet 77014d38e28345eda8f318ef76052807 (table test-table [id=b592f31e210849a7b3ff9e1bf338b7f5]) successfully deleted
I20260812 05:25:10.306569 26566 ts_tablet_manager.cc:1905] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:10.308779 26566 ts_tablet_manager.cc:1918] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:10.308881 26566 log.cc:1196] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/77014d38e28345eda8f318ef76052807
I20260812 05:25:10.309089 26566 ts_tablet_manager.cc:1939] T 77014d38e28345eda8f318ef76052807 P 2fdd51a7c4cc409ebbed107e87774afb: Deleting consensus metadata
Aug 12 05:25:10 dist-test-slave-zpfg krb5kdc[25964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512309, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:25:10.309494 25995 catalog_manager.cc:4841] TS 2fdd51a7c4cc409ebbed107e87774afb (127.24.66.67:40419): tablet 77014d38e28345eda8f318ef76052807 (table test-table [id=b592f31e210849a7b3ff9e1bf338b7f5]) successfully deleted
I20260812 05:25:10.313939 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.301933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.24.66.65:36749)
0812 05:25:10.301977 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:10.301980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:10.302360 (+ 380us) server_negotiation.cc:407] Connection header received
0812 05:25:10.302395 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:10.302396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:10.302411 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:10.302486 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:10.303235 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.304062 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.305145 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.305490 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.305569 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.310031 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:10.310045 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:10.310045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:10.310054 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:10.311903 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:10.312291 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:10.312295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:10.312295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:10.312365 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:10.312577 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:10.312577 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:10.312578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:10.312744 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:10.313756 (+ 1012us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:10.313798 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 05:25:10.313874 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 05:25:10.314587 26216 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49134
I20260812 05:25:10.322546 26062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:10.315685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57306 (local address 127.24.66.126:46175)
0812 05:25:10.315730 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:10.315732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:10.315835 (+ 103us) server_negotiation.cc:407] Connection header received
0812 05:25:10.315912 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:10.315913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:10.315923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:10.315973 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:10.316496 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.317000 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.317787 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:10.317990 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:10.318066 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:10.319533 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:10.319535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:10.319535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:10.319543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:10.321254 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:10.321704 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:10.321705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:10.321706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:10.321768 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:10.321987 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:10.321987 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:10.321988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:10.322106 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:10.322217 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:10.322320 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 05:25:10.322438 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 05:25:10.322831 26010 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57306
I20260812 05:25:10.323562 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 26087
I20260812 05:25:10.332849 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 26234
I20260812 05:25:10.340478 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 26384
I20260812 05:25:10.348076 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 25978
2026-08-12T05:25:10Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2494 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:10 dist-test-slave-zpfg krb5kdc[26606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:10 dist-test-slave-zpfg krb5kdc[26606](info): set up 2 sockets
Aug 12 05:25:10 dist-test-slave-zpfg krb5kdc[26606](info): commencing operation
krb5kdc: starting...
W20260812 05:25:11.471807 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:11 dist-test-slave-zpfg krb5kdc[26606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:25:11.614866 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40651
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:38517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:40651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:11.707341 26620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:11.707641 26620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:11.707690 26620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:11.709051 26620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:11.709142 26620 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 05:25:11.709167 26620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:11.709190 26620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:11.709239 26620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:11.709271 26620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:11.711234 26620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:38517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:40651
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:11.711573 26620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:11.711843 26620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:11.717530 26626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:11.717523 26625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:11.717523 26628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:11.718000 26620 server_base.cc:1034] running on GCE node
I20260812 05:25:11.718225 26620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:11.718554 26620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:11.719729 26620 hybrid_clock.cc:648] HybridClock initialized: now 1786512311719685 us; error 43 us; skew 500 ppm
Aug 12 05:25:11 dist-test-slave-zpfg krb5kdc[26606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512311, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:11.722626 26620 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:11.723089 26620 webserver.cc:460] Webserver started at http://127.24.66.126:34653/ using document root <none> and password file <none>
I20260812 05:25:11.723361 26620 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:11.723412 26620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:11.723591 26620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:11.724756 26620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "83022e460b084ea29390b9597f34994a"
format_stamp: "Formatted at 2026-08-12 05:25:11 on dist-test-slave-zpfg"
server_key: "2031e6e4fd6eebfd98d9fc943d01b686"
server_key_iv: "d9f1fcc2b4373396d9f360c7b3b1fe8e"
server_key_version: "encryptionkey@0"
I20260812 05:25:11.725179 26620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "83022e460b084ea29390b9597f34994a"
format_stamp: "Formatted at 2026-08-12 05:25:11 on dist-test-slave-zpfg"
server_key: "2031e6e4fd6eebfd98d9fc943d01b686"
server_key_iv: "d9f1fcc2b4373396d9f360c7b3b1fe8e"
server_key_version: "encryptionkey@0"
I20260812 05:25:11.726738 26620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260812 05:25:11.727595 26635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:11.727794 26620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:11.727919 26620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "83022e460b084ea29390b9597f34994a"
format_stamp: "Formatted at 2026-08-12 05:25:11 on dist-test-slave-zpfg"
server_key: "2031e6e4fd6eebfd98d9fc943d01b686"
server_key_iv: "d9f1fcc2b4373396d9f360c7b3b1fe8e"
server_key_version: "encryptionkey@0"
I20260812 05:25:11.728003 26620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:11.745402 26620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:11.748109 26620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:11.748308 26620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:11.753793 26620 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:40651
I20260812 05:25:11.753935 26700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:40651 every 8 connection(s)
I20260812 05:25:11.754264 26620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:11.755118 26701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:11.757861 26701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a: Bootstrap starting.
I20260812 05:25:11.758706 26701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:11.759027 26701 log.cc:824] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:11.759819 26701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a: No bootstrap required, opened a new log
I20260812 05:25:11.761503 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 26620
I20260812 05:25:11.761623 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:11.761833 24841 external_mini_cluster.cc:1428] Setting key 0a1bccced744c1d7b2f3d6be172b9cac
I20260812 05:25:11.763203 26701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83022e460b084ea29390b9597f34994a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40651 } }
I20260812 05:25:11.763398 26701 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:11.763442 26701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83022e460b084ea29390b9597f34994a, State: Initialized, Role: FOLLOWER
I20260812 05:25:11.763585 26701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83022e460b084ea29390b9597f34994a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40651 } }
I20260812 05:25:11.763716 26701 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:11.763744 26701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:11.763818 26701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:11.764588 26701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83022e460b084ea29390b9597f34994a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40651 } }
I20260812 05:25:11.764714 26701 leader_election.cc:302] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83022e460b084ea29390b9597f34994a; no voters:
I20260812 05:25:11.764889 26701 leader_election.cc:288] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:11.764954 26706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:11.765139 26706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [term 1 LEADER]: Becoming Leader. State: Replica: 83022e460b084ea29390b9597f34994a, State: Running, Role: LEADER
I20260812 05:25:11.765142 26701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:11.765264 26706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83022e460b084ea29390b9597f34994a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40651 } }
I20260812 05:25:11.766091 26707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83022e460b084ea29390b9597f34994a. Latest consensus state: current_term: 1 leader_uuid: "83022e460b084ea29390b9597f34994a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83022e460b084ea29390b9597f34994a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40651 } } }
I20260812 05:25:11.766201 26707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:11.766441 26715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:11.767019 26715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:11.768836 26715 catalog_manager.cc:1347] Generated new cluster ID: 27e6acea80224d4fab3b787ad9dbf63d
I20260812 05:25:11.768890 26715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:11.769586 26708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83022e460b084ea29390b9597f34994a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83022e460b084ea29390b9597f34994a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40651 } } }
I20260812 05:25:11.769690 26708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a [sys.catalog]: This master's current role is: LEADER
Aug 12 05:25:11 dist-test-slave-zpfg krb5kdc[26606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:11.774183 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:11.762575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52064 (local address 127.24.66.126:40651)
0812 05:25:11.762809 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:11.762821 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:11.762862 (+ 41us) server_negotiation.cc:407] Connection header received
0812 05:25:11.763231 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:11.763242 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:11.763313 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:11.763477 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:11.764209 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:11.765625 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:11.766887 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:11.767448 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:11.767513 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:11.770593 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:11.770598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:11.770599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:11.770613 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:11.772431 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:11.772834 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:11.772837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:11.772837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:11.772952 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:11.773177 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:11.773178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:11.773178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:11.773521 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:11.773693 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:11.773806 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 05:25:11.773979 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260812 05:25:11.788662 26715 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:11.789309 26715 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:11.791900 26715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a: Generated new TSK 0
I20260812 05:25:11.792150 26715 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:25:11.826203 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40651
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:38517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:11.921994 26729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:11.922291 26729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:11.922338 26729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:11.923619 26729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:11.923707 26729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:11.923743 26729 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 05:25:11.923771 26729 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:25:11.923813 26729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:11.925455 26729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:38517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.66.126:40651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:11.925779 26729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:11.926050 26729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:11.931718 26736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:11.931727 26738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:11.931742 26735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:11.932109 26729 server_base.cc:1034] running on GCE node
I20260812 05:25:11.932358 26729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:11.932657 26729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:11.933818 26729 hybrid_clock.cc:648] HybridClock initialized: now 1786512311933805 us; error 35 us; skew 500 ppm
Aug 12 05:25:11 dist-test-slave-zpfg krb5kdc[26606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512311, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:11.936628 26729 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:25:11.937187 26729 webserver.cc:460] Webserver started at http://127.24.66.65:36711/ using document root <none> and password file <none>
I20260812 05:25:11.937445 26729 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:11.937495 26729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:11.937655 26729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:11.938897 26729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "534fe3f7f4784a698217c859a828b21a"
format_stamp: "Formatted at 2026-08-12 05:25:11 on dist-test-slave-zpfg"
server_key: "97c6e294dae258260259501e0ada2a81"
server_key_iv: "e9a3160f8d8fb6264df6a703fe171895"
server_key_version: "encryptionkey@0"
I20260812 05:25:11.939275 26729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "534fe3f7f4784a698217c859a828b21a"
format_stamp: "Formatted at 2026-08-12 05:25:11 on dist-test-slave-zpfg"
server_key: "97c6e294dae258260259501e0ada2a81"
server_key_iv: "e9a3160f8d8fb6264df6a703fe171895"
server_key_version: "encryptionkey@0"
I20260812 05:25:11.940923 26729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:11.941831 26745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:11.942106 26729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:11.942219 26729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "534fe3f7f4784a698217c859a828b21a"
format_stamp: "Formatted at 2026-08-12 05:25:11 on dist-test-slave-zpfg"
server_key: "97c6e294dae258260259501e0ada2a81"
server_key_iv: "e9a3160f8d8fb6264df6a703fe171895"
server_key_version: "encryptionkey@0"
I20260812 05:25:11.942302 26729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:11.958432 26729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:11.960645 26729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:11.961061 26729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:11.961871 26729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:11.961938 26729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:11.961982 26729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:11.962003 26729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:25:11 dist-test-slave-zpfg krb5kdc[26606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512311, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:11.970832 26729 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:46009
I20260812 05:25:11.970959 26871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:46009 every 8 connection(s)
I20260812 05:25:11.971400 26729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:11.973333 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 26729
I20260812 05:25:11.973419 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:11.973618 24841 external_mini_cluster.cc:1428] Setting key bdecc8bef0c8720c28737a3420f000ab
I20260812 05:25:11.973774 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:11.962313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:36517 (local address 127.24.66.126:40651)
0812 05:25:11.962419 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:11.962421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:11.962811 (+ 390us) server_negotiation.cc:407] Connection header received
0812 05:25:11.963549 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:11.963551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:11.963568 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:11.963712 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:11.964738 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:11.965364 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:11.966721 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:11.967110 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:11.967190 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:11.970086 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:11.970088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:11.970088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:11.970101 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:11.972094 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:11.972552 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:11.972553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:11.972553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:11.972612 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:11.973009 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:11.973009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:11.973010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:11.973172 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:11.973249 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:11.973571 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 05:25:11.973681 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 05:25:11.974033 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:11.962302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40651 (local address 127.24.66.65:36517)
0812 05:25:11.962563 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:11.962659 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:11.963406 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:11.963812 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:11.963818 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:11.964268 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:11.964623 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:11.964631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:11.965485 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:11.965486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:11.966616 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:11.966619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:11.967233 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:11.967234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:11.967394 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:11.968020 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:11.968029 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:11.969983 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:11.972186 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:11.972192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:11.972195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:11.972472 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:11.972766 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:11.972768 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:11.972768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:11.972937 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:11.973244 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:11.973249 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:11.973380 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 05:25:11.973538 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 05:25:11.973720 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":162,"threads_started":1}
I20260812 05:25:11.976517 26872 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40651
I20260812 05:25:11.976637 26872 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:11.976879 26872 heartbeater.cc:511] Master 127.24.66.126:40651 requested a full tablet report, sending...
I20260812 05:25:11.977449 26652 ts_manager.cc:194] Registered new tserver with Master: 534fe3f7f4784a698217c859a828b21a (127.24.66.65:46009)
I20260812 05:25:11.978410 26652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:36517
I20260812 05:25:11.983299 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:11.975926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:42975 (local address 127.24.66.126:40651)
0812 05:25:11.976001 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:11.976003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:11.976256 (+ 253us) server_negotiation.cc:407] Connection header received
0812 05:25:11.976291 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:11.976292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:11.976307 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:11.976389 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:11.976780 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:11.977502 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:11.978600 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:11.978883 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:11.978934 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:11.980237 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:11.980239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:11.980239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:11.980246 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:11.981841 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:11.982299 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:11.982300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:11.982301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:11.982453 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:11.982710 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:11.982710 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:11.982711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:11.982843 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:11.982907 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:11.983113 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 05:25:11.983221 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:25:11.983383 26879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:11.975848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40651 (local address 127.24.66.65:42975)
0812 05:25:11.976010 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:11.976023 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:11.976103 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:11.976407 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:11.976408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:11.976631 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:11.976710 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:11.976713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:11.977619 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:11.977620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:11.978496 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:11.978499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:11.979024 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:11.979026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:11.979114 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:11.979649 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:11.979654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:11.980121 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:11.981941 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:11.981942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:11.981943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:11.982195 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:11.982545 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:11.982546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:11.982546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:11.982622 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:11.982924 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:11.982925 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:11.982989 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 05:25:11.983075 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 05:25:11.983212 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:25:12.005755 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40651
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:38517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:12.097254 26883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:12.097483 26883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:12.097510 26883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:12.098726 26883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:12.098771 26883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:12.098798 26883 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 05:25:12.098811 26883 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:25:12.098829 26883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:12.100689 26883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:38517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.66.126:40651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:12.100904 26883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:12.101097 26883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:12.106848 26889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:12.106781 26891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:12.107127 26883 server_base.cc:1034] running on GCE node
W20260812 05:25:12.106781 26888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:12.107419 26883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:12.107695 26883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:12.108902 26883 hybrid_clock.cc:648] HybridClock initialized: now 1786512312108905 us; error 53 us; skew 500 ppm
Aug 12 05:25:12 dist-test-slave-zpfg krb5kdc[26606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512312, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:12.116466 26883 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:25:12.117038 26883 webserver.cc:460] Webserver started at http://127.24.66.66:40709/ using document root <none> and password file <none>
I20260812 05:25:12.117290 26883 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:12.117363 26883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:12.117518 26883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:12.118717 26883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "52dd10d3e5c4445f82eecd19bbcb39dd"
format_stamp: "Formatted at 2026-08-12 05:25:12 on dist-test-slave-zpfg"
server_key: "12e865069a0208dd15ec4f50a794a396"
server_key_iv: "eab5d01733ff0e304a520a03e21e7de0"
server_key_version: "encryptionkey@0"
I20260812 05:25:12.119092 26883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "52dd10d3e5c4445f82eecd19bbcb39dd"
format_stamp: "Formatted at 2026-08-12 05:25:12 on dist-test-slave-zpfg"
server_key: "12e865069a0208dd15ec4f50a794a396"
server_key_iv: "eab5d01733ff0e304a520a03e21e7de0"
server_key_version: "encryptionkey@0"
I20260812 05:25:12.120510 26883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:12.121289 26899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:12.121555 26883 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:12.121693 26883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "52dd10d3e5c4445f82eecd19bbcb39dd"
format_stamp: "Formatted at 2026-08-12 05:25:12 on dist-test-slave-zpfg"
server_key: "12e865069a0208dd15ec4f50a794a396"
server_key_iv: "eab5d01733ff0e304a520a03e21e7de0"
server_key_version: "encryptionkey@0"
I20260812 05:25:12.121780 26883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:12.150507 26883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:12.152822 26883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:12.153003 26883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:12.153698 26883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:12.153753 26883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:12.153791 26883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:12.153836 26883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:25:12 dist-test-slave-zpfg krb5kdc[26606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512312, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:12.161899 26883 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:34699
I20260812 05:25:12.161979 27019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:34699 every 8 connection(s)
I20260812 05:25:12.162395 26883 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:12.162582 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 26883
I20260812 05:25:12.162694 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:12.162909 24841 external_mini_cluster.cc:1428] Setting key 38c24f2cb02822f73fc6657a8dbe89bc
I20260812 05:25:12.164635 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.154135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:54209 (local address 127.24.66.126:40651)
0812 05:25:12.154210 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.154213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.154476 (+ 263us) server_negotiation.cc:407] Connection header received
0812 05:25:12.155074 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.155077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.155092 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.155177 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:12.155818 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.156365 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.157469 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.157693 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.157774 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.161151 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:12.161153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:12.161154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:12.161162 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:12.163070 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.163514 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.163516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.163516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.163606 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.163855 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.163856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.163857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.164018 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:12.164103 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.164391 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.164499 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 05:25:12.164667 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.154132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40651 (local address 127.24.66.66:54209)
0812 05:25:12.154318 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.154394 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.154956 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.155250 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.155254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.155507 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:12.155707 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.155711 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.156471 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.156473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.157364 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.157367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.157782 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.157783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.157935 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.158617 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:12.158630 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:12.161030 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:12.163153 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:12.163158 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:12.163160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:12.163420 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:12.163671 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:12.163672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:12.163672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:12.163786 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:12.164097 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:12.164101 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:12.164227 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.164345 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.164496 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":116,"threads_started":1}
I20260812 05:25:12.165313 27020 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40651
I20260812 05:25:12.165414 27020 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:12.165622 27020 heartbeater.cc:511] Master 127.24.66.126:40651 requested a full tablet report, sending...
I20260812 05:25:12.166008 26652 ts_manager.cc:194] Registered new tserver with Master: 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699)
I20260812 05:25:12.166555 26652 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:54209
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
I20260812 05:25:12.172803 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.166455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40651 (local address 127.24.66.66:59839)
0812 05:25:12.166573 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.166585 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.166635 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.166903 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.166904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.167101 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:12.167170 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.167171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.167880 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.167881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.168434 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.168436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.168829 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.168830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.168912 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.169427 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:12.169432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:12.169770 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:12.171637 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:12.171638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:12.171639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:12.171846 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:12.172073 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:12.172073 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:12.172074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:12.172136 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:12.172431 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:12.172432 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:12.172484 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.172574 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.172683 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":6,"thread_start_us":64,"threads_started":1}
I20260812 05:25:12.172837 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.166701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:59839 (local address 127.24.66.126:40651)
0812 05:25:12.166768 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.166770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.166778 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:12.166798 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.166798 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.166808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.166856 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:12.167275 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.167777 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.168523 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.168740 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.168827 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.169914 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:12.169915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:12.169916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:12.169924 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:12.171514 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.171926 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.171927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.171928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.171991 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.172198 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.172199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.172199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.172357 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:12.172439 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.172600 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.172682 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:25:12.193495 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40651
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:38517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:12.283011 27032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:12.283253 27032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:12.283281 27032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:12.284512 27032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:12.284566 27032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:12.284590 27032 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 05:25:12.284639 27032 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:25:12.284667 27032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:12.286257 27032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:38517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.66.126:40651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:12.286584 27032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:12.286849 27032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:25:12.292614 27032 server_base.cc:1034] running on GCE node
W20260812 05:25:12.292725 27037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:12.292598 27038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:12.292584 27040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:12.293066 27032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:12.293344 27032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:12.294464 27032 hybrid_clock.cc:648] HybridClock initialized: now 1786512312294440 us; error 35 us; skew 500 ppm
Aug 12 05:25:12 dist-test-slave-zpfg krb5kdc[26606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512312, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:12.297263 27032 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:25:12.297976 27032 webserver.cc:460] Webserver started at http://127.24.66.67:35123/ using document root <none> and password file <none>
I20260812 05:25:12.298280 27032 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:12.298401 27032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:12.298556 27032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:12.300113 27032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "4de34691b57f4ca8a6e7760c2a3a8bde"
format_stamp: "Formatted at 2026-08-12 05:25:12 on dist-test-slave-zpfg"
server_key: "b8b1a122ed0ecf1d32705ec5736ae4b1"
server_key_iv: "1c355da407d360cc05f7957d24ea01c1"
server_key_version: "encryptionkey@0"
I20260812 05:25:12.300551 27032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "4de34691b57f4ca8a6e7760c2a3a8bde"
format_stamp: "Formatted at 2026-08-12 05:25:12 on dist-test-slave-zpfg"
server_key: "b8b1a122ed0ecf1d32705ec5736ae4b1"
server_key_iv: "1c355da407d360cc05f7957d24ea01c1"
server_key_version: "encryptionkey@0"
I20260812 05:25:12.302194 27032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:12.303289 27047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:12.303563 27032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:12.303653 27032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "4de34691b57f4ca8a6e7760c2a3a8bde"
format_stamp: "Formatted at 2026-08-12 05:25:12 on dist-test-slave-zpfg"
server_key: "b8b1a122ed0ecf1d32705ec5736ae4b1"
server_key_iv: "1c355da407d360cc05f7957d24ea01c1"
server_key_version: "encryptionkey@0"
I20260812 05:25:12.303717 27032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:12.329222 27032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:12.331930 27032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:12.332116 27032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:12.332756 27032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:12.332819 27032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:12.332856 27032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:12.332876 27032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:25:12 dist-test-slave-zpfg krb5kdc[26606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512312, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:12.340901 27032 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:43307
I20260812 05:25:12.341336 27032 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:12.342172 27171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:43307 every 8 connection(s)
I20260812 05:25:12.343241 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.333169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:53461 (local address 127.24.66.126:40651)
0812 05:25:12.333242 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.333244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.333492 (+ 248us) server_negotiation.cc:407] Connection header received
0812 05:25:12.333983 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.333985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.334000 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.334084 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:12.334972 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.335622 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.336663 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.336879 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.336966 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.339856 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:12.339858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:12.339859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:12.339867 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:12.341632 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.342138 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.342139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.342140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.342201 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.342534 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.342535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.342535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.342676 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:12.342754 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.343046 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.343151 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:25:12.343413 27057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.333168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40651 (local address 127.24.66.67:53461)
0812 05:25:12.333346 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.333389 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.333885 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.334156 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.334160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.334647 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:12.334853 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.334859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.335719 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.335720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.336563 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.336565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.336980 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.336981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.337185 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.337755 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:12.337774 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:12.339744 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:12.341780 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:12.341784 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:12.341786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:12.342060 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:12.342297 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:12.342298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:12.342298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:12.342451 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:12.342754 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:12.342762 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:12.342905 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.343003 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.343176 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260812 05:25:12.343933 27172 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40651
I20260812 05:25:12.344035 27172 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:12.344257 27172 heartbeater.cc:511] Master 127.24.66.126:40651 requested a full tablet report, sending...
I20260812 05:25:12.344650 26653 ts_manager.cc:194] Registered new tserver with Master: 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67:43307)
I20260812 05:25:12.345218 26653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:53461
I20260812 05:25:12.349807 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 27032
I20260812 05:25:12.349920 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:12.350134 24841 external_mini_cluster.cc:1428] Setting key 929b8b08c724e537185a74ef5940ce9b
I20260812 05:25:12.351356 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:25:12.351913 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.345065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40651 (local address 127.24.66.67:50295)
0812 05:25:12.345313 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.345326 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.345431 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.345576 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.345576 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.345847 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:12.345944 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.345947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.346872 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.346873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.347369 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.347370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.347755 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.347756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.347815 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.348378 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:12.348383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:12.348702 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:12.350730 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:12.350732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:12.350732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:12.350983 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:12.351264 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:12.351265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:12.351266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:12.351348 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:12.351635 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:12.351636 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:12.351683 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.351741 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.351834 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":73,"threads_started":1}
I20260812 05:25:12.351972 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.345123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:50295 (local address 127.24.66.126:40651)
0812 05:25:12.345163 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.345165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.345403 (+ 238us) server_negotiation.cc:407] Connection header received
0812 05:25:12.345489 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.345490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.345500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.345558 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:12.346057 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.346783 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.347484 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.347680 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.347745 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.348806 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:12.348808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:12.348808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:12.348815 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:12.350618 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.351060 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.351060 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.351061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.351144 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.351426 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.351427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.351427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.351567 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:12.351636 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.351787 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.351866 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 12 05:25:12 dist-test-slave-zpfg krb5kdc[26606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 05:25:12 dist-test-slave-zpfg krb5kdc[26606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:12.377713 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.369928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52076 (local address 127.24.66.126:40651)
0812 05:25:12.369993 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.369995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.370003 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:12.370031 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.370031 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.370041 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.370102 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:12.370849 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.371341 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.372189 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.372391 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.372453 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.375017 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:12.375019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:12.375019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:12.375027 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:12.376631 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.376985 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.376986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.376986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.377041 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.377274 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.377275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.377275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.377400 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:12.377484 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.377581 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.377642 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 05:25:12.379716 26653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:12.381407 26653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:12.389241 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.384172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:43307 (local address 127.0.0.1:43928)
0812 05:25:12.384672 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.384685 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.384732 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.385628 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.385629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.385631 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:12.385673 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.385674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.386926 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.386927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.387840 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.387843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.388838 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.388839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.388943 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.388961 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.389049 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.389116 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":4,"thread_start_us":75,"threads_started":1}
I20260812 05:25:12.389544 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.384875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37942 (local address 127.24.66.65:46009)
0812 05:25:12.385064 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.385069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.385081 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:25:12.385131 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.385133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.385285 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.385441 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:12.385567 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.386682 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.388123 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.388860 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.388939 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.388977 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.389383 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.389441 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260812 05:25:12.389235 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.385104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43928 (local address 127.24.66.67:43307)
0812 05:25:12.385396 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.385400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.385411 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:12.385452 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.385455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.385471 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.385557 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:12.385764 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.386819 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.387953 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.388703 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.388786 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.388852 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.389075 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.389119 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":95,"threads_started":1}
I20260812 05:25:12.389241 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.384543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:46009 (local address 127.0.0.1:37942)
0812 05:25:12.384718 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.384728 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.384770 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.385447 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.385448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.385451 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:12.385506 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.385508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.386781 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.386782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.388025 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.388027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.388948 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.388949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.389013 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.389017 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.389074 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.389125 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":43,"thread_start_us":117,"threads_started":1}
I20260812 05:25:12.390162 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.384172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:34699 (local address 127.0.0.1:57566)
0812 05:25:12.384378 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.384427 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.384507 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.385149 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.385153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.385156 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:12.385263 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.385266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.386524 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.386525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.388022 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.388025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.389915 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.389916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.389983 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.389987 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.390043 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.390090 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":134,"threads_started":1}
I20260812 05:25:12.390739 27100 tablet_service.cc:1467] Processing CreateTablet for tablet 882e9df8ea704672af9f2623400d4cd8 (DEFAULT_TABLE table=test-table [id=e6bb836e7e37465793d1f37b73d1d04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:12.390738 26801 tablet_service.cc:1467] Processing CreateTablet for tablet 882e9df8ea704672af9f2623400d4cd8 (DEFAULT_TABLE table=test-table [id=e6bb836e7e37465793d1f37b73d1d04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:12.391115 26801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882e9df8ea704672af9f2623400d4cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:12.391127 27100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882e9df8ea704672af9f2623400d4cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:12.394160 27196 tablet_bootstrap.cc:492] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde: Bootstrap starting.
I20260812 05:25:12.394488 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.384657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.24.66.66:34699)
0812 05:25:12.384886 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.384892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.384905 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:25:12.384995 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.384997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.385017 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.385365 (+ 348us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:12.385438 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.386421 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.389014 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.389815 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.389898 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.389955 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.390056 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.390094 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":109,"threads_started":1}
I20260812 05:25:12.394536 27197 tablet_bootstrap.cc:492] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: Bootstrap starting.
I20260812 05:25:12.395278 27196 tablet_bootstrap.cc:654] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:12.395304 27197 tablet_bootstrap.cc:654] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:12.395268 26950 tablet_service.cc:1467] Processing CreateTablet for tablet 882e9df8ea704672af9f2623400d4cd8 (DEFAULT_TABLE table=test-table [id=e6bb836e7e37465793d1f37b73d1d04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:12.395661 27196 log.cc:824] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:12.395722 26950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882e9df8ea704672af9f2623400d4cd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:12.395713 27197 log.cc:824] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:12.396703 27196 tablet_bootstrap.cc:492] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde: No bootstrap required, opened a new log
I20260812 05:25:12.396797 27196 ts_tablet_manager.cc:1397] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:25:12.396859 27197 tablet_bootstrap.cc:492] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: No bootstrap required, opened a new log
I20260812 05:25:12.396947 27197 ts_tablet_manager.cc:1397] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:25:12.398331 27196 raft_consensus.cc:348] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.398586 27196 raft_consensus.cc:374] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:12.398694 27196 raft_consensus.cc:729] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de34691b57f4ca8a6e7760c2a3a8bde, State: Initialized, Role: FOLLOWER
I20260812 05:25:12.398841 27196 consensus_queue.cc:260] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.399186 27196 ts_tablet_manager.cc:1428] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 05:25:12.399389 27197 raft_consensus.cc:348] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.399570 27197 raft_consensus.cc:374] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:12.399618 27197 raft_consensus.cc:729] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 534fe3f7f4784a698217c859a828b21a, State: Initialized, Role: FOLLOWER
I20260812 05:25:12.399766 27197 consensus_queue.cc:260] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.400072 27197 ts_tablet_manager.cc:1428] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 05:25:12.400272 27172 heartbeater.cc:503] Master 127.24.66.126:40651 was elected leader, sending a full tablet report...
I20260812 05:25:12.400494 26872 heartbeater.cc:503] Master 127.24.66.126:40651 was elected leader, sending a full tablet report...
I20260812 05:25:12.400520 27201 tablet_bootstrap.cc:492] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: Bootstrap starting.
I20260812 05:25:12.401293 27201 tablet_bootstrap.cc:654] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:12.401702 27201 log.cc:824] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:12.402729 27201 tablet_bootstrap.cc:492] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: No bootstrap required, opened a new log
I20260812 05:25:12.402807 27201 ts_tablet_manager.cc:1397] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:25:12.404193 27201 raft_consensus.cc:348] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.404337 27201 raft_consensus.cc:374] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:12.404361 27201 raft_consensus.cc:729] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dd10d3e5c4445f82eecd19bbcb39dd, State: Initialized, Role: FOLLOWER
I20260812 05:25:12.404450 27201 consensus_queue.cc:260] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.404706 27201 ts_tablet_manager.cc:1428] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:25:12.404835 27020 heartbeater.cc:503] Master 127.24.66.126:40651 was elected leader, sending a full tablet report...
W20260812 05:25:12.413378 27021 tablet.cc:2369] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:25:12.452457 27202 raft_consensus.cc:482] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:12.452801 27202 raft_consensus.cc:504] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.453270 27202 leader_election.cc:288] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699), 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67:43307)
I20260812 05:25:12.456645 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.453639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:41069 (local address 127.24.66.67:43307)
0812 05:25:12.453686 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.453689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.453843 (+ 154us) server_negotiation.cc:407] Connection header received
0812 05:25:12.453910 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.453911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.453921 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.454026 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:12.454210 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.454834 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.455844 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.456275 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.456367 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.456377 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.456538 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.456579 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:25:12.456678 27205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.453546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:43307 (local address 127.24.66.65:41069)
0812 05:25:12.453752 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.453769 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.453853 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.454035 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.454036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.454038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:12.454134 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.454136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.454919 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.454920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.455738 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.455741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.456368 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.456369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.456445 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.456449 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.456519 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.456564 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260812 05:25:12.456771 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.453537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:48803 (local address 127.24.66.66:34699)
0812 05:25:12.453589 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.453591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.453728 (+ 137us) server_negotiation.cc:407] Connection header received
0812 05:25:12.453770 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.453772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.453790 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.453866 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:12.454052 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.454759 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.455868 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.456333 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.456432 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.456442 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.456630 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.456684 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:25:12.456997 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.453485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:34699 (local address 127.24.66.65:48803)
0812 05:25:12.453560 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.453576 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.453688 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.453869 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.453870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.453871 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:12.453964 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.453966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.454849 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.454850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.455745 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.455748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.456432 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.456433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.456526 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.456530 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.456620 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.456675 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":5}
I20260812 05:25:12.457281 27121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882e9df8ea704672af9f2623400d4cd8" candidate_uuid: "534fe3f7f4784a698217c859a828b21a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" is_pre_election: true
I20260812 05:25:12.457381 26971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882e9df8ea704672af9f2623400d4cd8" candidate_uuid: "534fe3f7f4784a698217c859a828b21a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" is_pre_election: true
I20260812 05:25:12.457490 27121 raft_consensus.cc:2454] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 534fe3f7f4784a698217c859a828b21a in term 0.
I20260812 05:25:12.457572 26971 raft_consensus.cc:2454] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 534fe3f7f4784a698217c859a828b21a in term 0.
I20260812 05:25:12.457792 26747 leader_election.cc:302] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4de34691b57f4ca8a6e7760c2a3a8bde, 534fe3f7f4784a698217c859a828b21a; no voters:
I20260812 05:25:12.457965 27202 raft_consensus.cc:2781] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:12.458047 27202 raft_consensus.cc:482] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:12.458107 27202 raft_consensus.cc:3037] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:12.459132 27202 raft_consensus.cc:504] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.459506 27202 leader_election.cc:288] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [CANDIDATE]: Term 1 election: Requested vote from peers 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699), 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67:43307)
I20260812 05:25:12.459645 26971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882e9df8ea704672af9f2623400d4cd8" candidate_uuid: "534fe3f7f4784a698217c859a828b21a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd"
I20260812 05:25:12.459704 27121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882e9df8ea704672af9f2623400d4cd8" candidate_uuid: "534fe3f7f4784a698217c859a828b21a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde"
I20260812 05:25:12.459746 26971 raft_consensus.cc:3037] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:12.459828 27121 raft_consensus.cc:3037] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:12.460536 26971 raft_consensus.cc:2454] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 534fe3f7f4784a698217c859a828b21a in term 1.
I20260812 05:25:12.460772 26748 leader_election.cc:302] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52dd10d3e5c4445f82eecd19bbcb39dd, 534fe3f7f4784a698217c859a828b21a; no voters:
I20260812 05:25:12.460803 27121 raft_consensus.cc:2454] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 534fe3f7f4784a698217c859a828b21a in term 1.
I20260812 05:25:12.460896 27202 raft_consensus.cc:2781] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:12.461095 27202 raft_consensus.cc:686] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 1 LEADER]: Becoming Leader. State: Replica: 534fe3f7f4784a698217c859a828b21a, State: Running, Role: LEADER
I20260812 05:25:12.461236 27202 consensus_queue.cc:237] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.462218 26653 catalog_manager.cc:5495] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a reported cstate change: term changed from 0 to 1, leader changed from <none> to 534fe3f7f4784a698217c859a828b21a (127.24.66.65). New cstate: current_term: 1 leader_uuid: "534fe3f7f4784a698217c859a828b21a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } health_report { overall_health: UNKNOWN } } }
W20260812 05:25:12.472548 26873 tablet.cc:2369] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 05:25:12 dist-test-slave-zpfg krb5kdc[26606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512311, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:12.507473 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.499356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52082 (local address 127.24.66.126:40651)
0812 05:25:12.499477 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.499479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.499533 (+ 54us) server_negotiation.cc:407] Connection header received
0812 05:25:12.499593 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.499594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.499604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.499704 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:12.500319 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.500824 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.501656 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.501860 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.501928 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.504354 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:12.504355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:12.504355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:12.504362 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:12.505820 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.506297 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.506297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.506298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.506444 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:12.506714 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:12.506715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:12.506716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:12.506854 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:12.506967 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.507238 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.507356 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 05:25:12.507550 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.499256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40651 (local address 127.0.0.1:52082)
0812 05:25:12.499452 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.499463 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.499536 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.499741 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.499743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.500155 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:12.500211 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.500214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.500948 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.500949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.501536 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.501538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.501946 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.501947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.501998 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.502672 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:12.502677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:12.504203 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:12.505911 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:12.505919 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:12.505921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:12.506221 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:12.506504 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:12.506505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:12.506505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:12.506568 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:12.506962 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:12.506972 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:12.507019 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.507209 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.507426 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":97,"thread_start_us":92,"threads_started":1}
I20260812 05:25:12.510272 26653 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:52082:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 05:25:12.510600 26653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:12.512655 26801 tablet_service.cc:1467] Processing CreateTablet for tablet 420ac61428ba4ab095c63363b99066de (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8cd1f8ed10c84fcab2e1a2c11e8aaf61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:25:12.512735 26950 tablet_service.cc:1467] Processing CreateTablet for tablet 420ac61428ba4ab095c63363b99066de (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8cd1f8ed10c84fcab2e1a2c11e8aaf61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:25:12.512782 27100 tablet_service.cc:1467] Processing CreateTablet for tablet 420ac61428ba4ab095c63363b99066de (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8cd1f8ed10c84fcab2e1a2c11e8aaf61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:25:12.512877 26801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420ac61428ba4ab095c63363b99066de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:12.512935 27100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420ac61428ba4ab095c63363b99066de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:12.512940 26950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420ac61428ba4ab095c63363b99066de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:12.514633 27201 tablet_bootstrap.cc:492] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: Bootstrap starting.
I20260812 05:25:12.514672 27197 tablet_bootstrap.cc:492] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a: Bootstrap starting.
I20260812 05:25:12.514681 27196 tablet_bootstrap.cc:492] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: Bootstrap starting.
I20260812 05:25:12.515214 27197 tablet_bootstrap.cc:654] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:12.515290 27196 tablet_bootstrap.cc:654] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:12.515342 27201 tablet_bootstrap.cc:654] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:12.516047 27197 tablet_bootstrap.cc:492] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a: No bootstrap required, opened a new log
I20260812 05:25:12.516132 27197 ts_tablet_manager.cc:1397] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:12.516252 27196 tablet_bootstrap.cc:492] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: No bootstrap required, opened a new log
I20260812 05:25:12.516336 27196 ts_tablet_manager.cc:1397] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:25:12.516311 27197 raft_consensus.cc:348] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.516391 27197 raft_consensus.cc:374] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:12.516412 27197 raft_consensus.cc:729] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 534fe3f7f4784a698217c859a828b21a, State: Initialized, Role: FOLLOWER
I20260812 05:25:12.516474 27197 consensus_queue.cc:260] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.516490 27201 tablet_bootstrap.cc:492] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: No bootstrap required, opened a new log
I20260812 05:25:12.516538 27201 ts_tablet_manager.cc:1397] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:25:12.516510 27196 raft_consensus.cc:348] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.516587 27197 ts_tablet_manager.cc:1428] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:12.516602 27196 raft_consensus.cc:374] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:12.516645 27196 raft_consensus.cc:729] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de34691b57f4ca8a6e7760c2a3a8bde, State: Initialized, Role: FOLLOWER
I20260812 05:25:12.516692 27196 consensus_queue.cc:260] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.516708 27201 raft_consensus.cc:348] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.516780 27201 raft_consensus.cc:374] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:12.516803 27201 raft_consensus.cc:729] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dd10d3e5c4445f82eecd19bbcb39dd, State: Initialized, Role: FOLLOWER
I20260812 05:25:12.516803 27196 ts_tablet_manager.cc:1428] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:12.516855 27201 consensus_queue.cc:260] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.516894 27200 tablet_replica.cc:439] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } } }
I20260812 05:25:12.516971 27201 ts_tablet_manager.cc:1428] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:12.516995 27200 tablet_replica.cc:442] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:12.517048 27204 tablet_replica.cc:439] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } } }
I20260812 05:25:12.517158 27204 tablet_replica.cc:442] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:12.517103 27202 tablet_replica.cc:439] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } } }
I20260812 05:25:12.517213 27202 tablet_replica.cc:442] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 05:25:12.591996 27173 tablet.cc:2369] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:25:12.633029 27204 raft_consensus.cc:482] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:12.633153 27204 raft_consensus.cc:504] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.633623 27204 leader_election.cc:288] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 534fe3f7f4784a698217c859a828b21a (127.24.66.65:46009), 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67:43307)
I20260812 05:25:12.637070 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.633897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:46009 (local address 127.24.66.66:52259)
0812 05:25:12.634060 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.634072 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.634125 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.634299 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.634301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.634303 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:12.634430 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.634432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.635206 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.635208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.636111 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.636113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.636752 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.636753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.636838 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.636841 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.636915 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.636964 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":106,"threads_started":1}
I20260812 05:25:12.637060 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.634087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:52259 (local address 127.24.66.65:46009)
0812 05:25:12.634157 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.634160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.634171 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:12.634201 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.634202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.634211 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.634295 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:12.634524 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.635097 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.636222 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.636637 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.636731 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.636753 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.636938 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.636980 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 05:25:12.637287 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.633944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:44763 (local address 127.24.66.67:43307)
0812 05:25:12.634009 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.634011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.634023 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:25:12.634123 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.634125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.634137 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.634202 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:12.634607 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.635100 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.636417 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.636879 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.636973 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.636983 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.637165 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.637214 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 05:25:12.637287 26908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.633848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:43307 (local address 127.24.66.66:44763)
0812 05:25:12.633941 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.633954 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.634028 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.634325 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.634326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.634328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:12.634423 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.634425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.635326 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.635328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.636289 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.636292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.636984 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.636986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.637082 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.637085 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.637159 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.637210 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 05:25:12.637588 27121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420ac61428ba4ab095c63363b99066de" candidate_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" is_pre_election: true
I20260812 05:25:12.637708 27121 raft_consensus.cc:2454] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52dd10d3e5c4445f82eecd19bbcb39dd in term 0.
I20260812 05:25:12.637676 26822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420ac61428ba4ab095c63363b99066de" candidate_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "534fe3f7f4784a698217c859a828b21a" is_pre_election: true
I20260812 05:25:12.637858 26822 raft_consensus.cc:2454] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52dd10d3e5c4445f82eecd19bbcb39dd in term 0.
I20260812 05:25:12.637926 26901 leader_election.cc:302] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4de34691b57f4ca8a6e7760c2a3a8bde, 52dd10d3e5c4445f82eecd19bbcb39dd; no voters:
I20260812 05:25:12.638104 27204 raft_consensus.cc:2781] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:12.638178 27204 raft_consensus.cc:482] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:12.638208 27204 raft_consensus.cc:3037] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:12.638999 27204 raft_consensus.cc:504] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.639192 27204 leader_election.cc:288] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [CANDIDATE]: Term 1 election: Requested vote from peers 534fe3f7f4784a698217c859a828b21a (127.24.66.65:46009), 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67:43307)
I20260812 05:25:12.639380 26822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420ac61428ba4ab095c63363b99066de" candidate_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "534fe3f7f4784a698217c859a828b21a"
I20260812 05:25:12.639398 27121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420ac61428ba4ab095c63363b99066de" candidate_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde"
I20260812 05:25:12.639479 26822 raft_consensus.cc:3037] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:12.639479 27121 raft_consensus.cc:3037] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:12.640405 27121 raft_consensus.cc:2454] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52dd10d3e5c4445f82eecd19bbcb39dd in term 1.
I20260812 05:25:12.640417 26822 raft_consensus.cc:2454] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52dd10d3e5c4445f82eecd19bbcb39dd in term 1.
I20260812 05:25:12.640650 26902 leader_election.cc:302] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52dd10d3e5c4445f82eecd19bbcb39dd, 534fe3f7f4784a698217c859a828b21a; no voters:
I20260812 05:25:12.640784 27204 raft_consensus.cc:2781] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:12.640992 27204 raft_consensus.cc:686] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 LEADER]: Becoming Leader. State: Replica: 52dd10d3e5c4445f82eecd19bbcb39dd, State: Running, Role: LEADER
I20260812 05:25:12.641120 27204 consensus_queue.cc:237] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.641597 27218 tablet_replica.cc:439] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: TxnStatusTablet state changed. Reason: New leader 52dd10d3e5c4445f82eecd19bbcb39dd. Latest consensus state: current_term: 1 leader_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } } }
I20260812 05:25:12.641700 27218 tablet_replica.cc:442] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:25:12.641981 27220 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 05:25:12.642150 27220 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 05:25:12.642149 26651 catalog_manager.cc:5495] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66). New cstate: current_term: 1 leader_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } health_report { overall_health: UNKNOWN } } }
I20260812 05:25:12.667217 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.663805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57578 (local address 127.24.66.66:34699)
0812 05:25:12.663870 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.663872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.663882 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:12.663926 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.663927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.663950 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.664042 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:12.664172 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.665080 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.665944 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.666384 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.666470 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.666657 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:12.666809 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:12.666926 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.667045 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.667072 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 05:25:12.667232 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.663684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:34699 (local address 127.0.0.1:57578)
0812 05:25:12.663760 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.663774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.663829 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.664047 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.664048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.664049 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:25:12.664098 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.664100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.665162 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.665163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.665860 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.665862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.666467 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.666468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.666522 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.666546 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:25:12.666941 (+ 395us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:25:12.666963 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.667019 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.667103 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 05:25:12.669484 27121 raft_consensus.cc:1261] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Refusing update from remote peer 52dd10d3e5c4445f82eecd19bbcb39dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:12.669545 26822 raft_consensus.cc:1261] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Refusing update from remote peer 52dd10d3e5c4445f82eecd19bbcb39dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:12.669873 27218 consensus_queue.cc:1035] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:12.670090 27218 consensus_queue.cc:1035] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:12.671348 27202 tablet_replica.cc:439] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a: TxnStatusTablet state changed. Reason: New leader 52dd10d3e5c4445f82eecd19bbcb39dd. Latest consensus state: current_term: 1 leader_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } } }
I20260812 05:25:12.671486 27202 tablet_replica.cc:442] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:12.671779 27200 tablet_replica.cc:439] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: TxnStatusTablet state changed. Reason: New leader 52dd10d3e5c4445f82eecd19bbcb39dd. Latest consensus state: current_term: 1 leader_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } } }
I20260812 05:25:12.671895 27200 tablet_replica.cc:442] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:12.672851 27218 tablet_replica.cc:439] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } } }
I20260812 05:25:12.672945 27218 tablet_replica.cc:442] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:25:12.672995 27218 tablet_replica.cc:439] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } } }
I20260812 05:25:12.673041 27218 tablet_replica.cc:442] T 420ac61428ba4ab095c63363b99066de P 52dd10d3e5c4445f82eecd19bbcb39dd: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:25:12.673233 27202 tablet_replica.cc:439] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } } }
I20260812 05:25:12.673323 27202 tablet_replica.cc:442] T 420ac61428ba4ab095c63363b99066de P 534fe3f7f4784a698217c859a828b21a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:12.673661 27200 tablet_replica.cc:439] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } } }
I20260812 05:25:12.673751 27200 tablet_replica.cc:442] T 420ac61428ba4ab095c63363b99066de P 4de34691b57f4ca8a6e7760c2a3a8bde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:12.673981 27225 mvcc.cc:205] Tried to move back new op lower bound from 7317554432691171328 to 7317554432578568192. Current Snapshot: MvccSnapshot[applied={T|T < 7317554432691171328}]
I20260812 05:25:12.679122 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.676156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.24.66.65:46009)
0812 05:25:12.676212 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.676214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.676225 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:12.676308 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.676309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.676328 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.676402 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:12.676541 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.677372 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.678118 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.678511 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.678592 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.678718 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:12.678863 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:12.678955 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.678999 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.679023 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 05:25:12.685555 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.682034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:38139 (local address 127.24.66.66:34699)
0812 05:25:12.682074 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.682076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.682246 (+ 170us) server_negotiation.cc:407] Connection header received
0812 05:25:12.682314 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.682315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.682324 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.682416 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:12.682704 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.683161 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.684681 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.684861 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.684914 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.685236 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:12.685293 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:12.685333 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.685466 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.685497 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 05:25:12.685685 27235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.681966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:34699 (local address 127.24.66.65:38139)
0812 05:25:12.682172 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.682185 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.682251 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.682430 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.682430 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.682432 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:25:12.682630 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.682632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.683249 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.683250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.684561 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.684565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.684943 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.684945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.685064 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.685084 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:25:12.685365 (+ 281us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:25:12.685368 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.685463 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.685515 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":104,"threads_started":1}
I20260812 05:25:12.688216 27121 raft_consensus.cc:1261] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Refusing update from remote peer 534fe3f7f4784a698217c859a828b21a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:12.688324 26971 raft_consensus.cc:1261] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Refusing update from remote peer 534fe3f7f4784a698217c859a828b21a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:12.688620 27206 consensus_queue.cc:1035] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:12.688782 27202 consensus_queue.cc:1035] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:12.690310 27224 mvcc.cc:205] Tried to move back new op lower bound from 7317554432769331200 to 7317554431841951744. Current Snapshot: MvccSnapshot[applied={T|T < 7317554432769331200}]
I20260812 05:25:12.699848 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.696211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:33281 (local address 127.24.66.65:46009)
0812 05:25:12.696258 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.696261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.696480 (+ 219us) server_negotiation.cc:407] Connection header received
0812 05:25:12.696524 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.696525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.696538 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.696617 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:12.696758 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.697624 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.698907 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.699248 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.699332 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.699543 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:12.699580 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:12.699648 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.699737 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.699760 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:25:12.699968 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.696153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:46009 (local address 127.24.66.66:33281)
0812 05:25:12.696405 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.696415 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.696479 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.696611 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.696611 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.696613 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:25:12.696696 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.696697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.697724 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.697725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.698818 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.698822 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.699342 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.699343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.699439 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.699455 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:25:12.699681 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:25:12.699684 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.699730 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.699773 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":124,"threads_started":1}
I20260812 05:25:12.709933 26652 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:12.710063 26652 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:12.711014 26652 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a: Sending DeleteTablet for 3 replicas of tablet 882e9df8ea704672af9f2623400d4cd8
I20260812 05:25:12.711395 26801 tablet_service.cc:1514] Processing DeleteTablet for tablet 882e9df8ea704672af9f2623400d4cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:12 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:37942
I20260812 05:25:12.711781 27243 tablet_replica.cc:330] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: stopping tablet replica
I20260812 05:25:12.711773 26950 tablet_service.cc:1514] Processing DeleteTablet for tablet 882e9df8ea704672af9f2623400d4cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:12 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:57566
I20260812 05:25:12.712011 27243 raft_consensus.cc:2229] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:12.712210 27243 raft_consensus.cc:2258] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:12.712212 27100 tablet_service.cc:1514] Processing DeleteTablet for tablet 882e9df8ea704672af9f2623400d4cd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:12 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:43928
I20260812 05:25:12.712666 26652 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:12.712826 26652 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:12.713248 27244 tablet_replica.cc:330] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: stopping tablet replica
I20260812 05:25:12.713454 27244 raft_consensus.cc:2229] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:12.713531 27244 raft_consensus.cc:2258] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:12.714003 27244 ts_tablet_manager.cc:1905] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:12.714747 27100 tablet_service.cc:1467] Processing CreateTablet for tablet 08a1fa90c7cd44c9ba3e48633aff5a75 (DEFAULT_TABLE table=test-table [id=1c1133e9b7a9487bbfea43ef1ddb1475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:12.714888 27100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08a1fa90c7cd44c9ba3e48633aff5a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:12.714934 26950 tablet_service.cc:1467] Processing CreateTablet for tablet 08a1fa90c7cd44c9ba3e48633aff5a75 (DEFAULT_TABLE table=test-table [id=1c1133e9b7a9487bbfea43ef1ddb1475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:12.715066 26950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08a1fa90c7cd44c9ba3e48633aff5a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:12.716638 27201 tablet_bootstrap.cc:492] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd: Bootstrap starting.
I20260812 05:25:12.716740 27196 tablet_bootstrap.cc:492] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde: Bootstrap starting.
I20260812 05:25:12.714787 26801 tablet_service.cc:1467] Processing CreateTablet for tablet 08a1fa90c7cd44c9ba3e48633aff5a75 (DEFAULT_TABLE table=test-table [id=1c1133e9b7a9487bbfea43ef1ddb1475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:12.717453 27201 tablet_bootstrap.cc:654] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:12.717453 27196 tablet_bootstrap.cc:654] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:12.717465 26801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08a1fa90c7cd44c9ba3e48633aff5a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:12.719142 27197 tablet_bootstrap.cc:492] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a: Bootstrap starting.
I20260812 05:25:12.720113 27197 tablet_bootstrap.cc:654] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:12.720136 27244 ts_tablet_manager.cc:1918] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:12.720209 27244 log.cc:1196] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/882e9df8ea704672af9f2623400d4cd8
I20260812 05:25:12.720499 27244 ts_tablet_manager.cc:1939] T 882e9df8ea704672af9f2623400d4cd8 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting consensus metadata
I20260812 05:25:12.720981 26638 catalog_manager.cc:4841] TS 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699): tablet 882e9df8ea704672af9f2623400d4cd8 (table test-table [id=e6bb836e7e37465793d1f37b73d1d04e]) successfully deleted
I20260812 05:25:12.721297 27196 tablet_bootstrap.cc:492] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde: No bootstrap required, opened a new log
I20260812 05:25:12.721352 27196 ts_tablet_manager.cc:1397] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 05:25:12.713086 27243 ts_tablet_manager.cc:1905] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:12.721530 27196 raft_consensus.cc:348] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.721618 27196 raft_consensus.cc:374] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:12.721675 27196 raft_consensus.cc:729] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de34691b57f4ca8a6e7760c2a3a8bde, State: Initialized, Role: FOLLOWER
I20260812 05:25:12.721728 27196 consensus_queue.cc:260] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.721858 27196 ts_tablet_manager.cc:1428] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:25:12.722549 27245 tablet_replica.cc:330] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde: stopping tablet replica
I20260812 05:25:12.722739 27245 raft_consensus.cc:2229] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:12.722820 27245 raft_consensus.cc:2258] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:12.723388 27197 tablet_bootstrap.cc:492] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a: No bootstrap required, opened a new log
I20260812 05:25:12.723448 27197 ts_tablet_manager.cc:1397] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 05:25:12.723601 27197 raft_consensus.cc:348] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.723701 27197 raft_consensus.cc:374] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:12.723726 27197 raft_consensus.cc:729] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 534fe3f7f4784a698217c859a828b21a, State: Initialized, Role: FOLLOWER
I20260812 05:25:12.723773 27197 consensus_queue.cc:260] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.723902 27197 ts_tablet_manager.cc:1428] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:25:12.724174 27243 ts_tablet_manager.cc:1918] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:12.724264 27243 log.cc:1196] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/wal/wals/882e9df8ea704672af9f2623400d4cd8
I20260812 05:25:12.724319 27245 ts_tablet_manager.cc:1905] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:12.724613 27243 ts_tablet_manager.cc:1939] T 882e9df8ea704672af9f2623400d4cd8 P 534fe3f7f4784a698217c859a828b21a: Deleting consensus metadata
I20260812 05:25:12.725059 27201 tablet_bootstrap.cc:492] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd: No bootstrap required, opened a new log
I20260812 05:25:12.725154 27201 ts_tablet_manager.cc:1397] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260812 05:25:12.725703 27201 raft_consensus.cc:348] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.725808 27201 raft_consensus.cc:374] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:12.725833 27201 raft_consensus.cc:729] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dd10d3e5c4445f82eecd19bbcb39dd, State: Initialized, Role: FOLLOWER
I20260812 05:25:12.725900 27201 consensus_queue.cc:260] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.726040 27201 ts_tablet_manager.cc:1428] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:12.726712 26638 catalog_manager.cc:4841] TS 534fe3f7f4784a698217c859a828b21a (127.24.66.65:46009): tablet 882e9df8ea704672af9f2623400d4cd8 (table test-table [id=e6bb836e7e37465793d1f37b73d1d04e]) successfully deleted
I20260812 05:25:12.726886 27245 ts_tablet_manager.cc:1918] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:12.726956 27245 log.cc:1196] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/882e9df8ea704672af9f2623400d4cd8
I20260812 05:25:12.727172 27245 ts_tablet_manager.cc:1939] T 882e9df8ea704672af9f2623400d4cd8 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting consensus metadata
I20260812 05:25:12.727860 26637 catalog_manager.cc:4841] TS 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67:43307): tablet 882e9df8ea704672af9f2623400d4cd8 (table test-table [id=e6bb836e7e37465793d1f37b73d1d04e]) successfully deleted
I20260812 05:25:12.981405 27200 raft_consensus.cc:482] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:12.981523 27200 raft_consensus.cc:504] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.981848 27200 leader_election.cc:288] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 534fe3f7f4784a698217c859a828b21a (127.24.66.65:46009), 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699)
I20260812 05:25:12.985146 27250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.982262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:34699 (local address 127.24.66.67:38275)
0812 05:25:12.982363 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.982375 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.982456 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.982618 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.982619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.982621 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:12.982701 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.982702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.983421 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.983422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.984258 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.984260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.984845 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.984845 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.984928 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.984931 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.985001 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.985046 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":30,"thread_start_us":43,"threads_started":1}
I20260812 05:25:12.985126 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.982399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:38275 (local address 127.24.66.66:34699)
0812 05:25:12.982485 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.982488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.982497 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:12.982527 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.982528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.982540 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.982615 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:12.982775 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.983249 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.984370 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.984753 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.984844 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.984854 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.985009 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.985051 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 05:25:12.985443 26971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08a1fa90c7cd44c9ba3e48633aff5a75" candidate_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" is_pre_election: true
I20260812 05:25:12.985539 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.982135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:60859 (local address 127.24.66.65:46009)
0812 05:25:12.982218 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:12.982221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:12.982442 (+ 221us) server_negotiation.cc:407] Connection header received
0812 05:25:12.982486 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:12.982487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:12.982503 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:12.982578 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:12.982744 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.983282 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.984521 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.984939 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.985042 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.985052 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:12.985424 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 05:25:12.985466 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:25:12.985556 27249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:12.982029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:46009 (local address 127.24.66.67:60859)
0812 05:25:12.982216 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:12.982230 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:12.982389 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:12.982583 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:12.982584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:12.982588 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:12.982675 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.982678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.983377 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.983378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.984406 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:12.984409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:12.985045 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:12.985046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:12.985137 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:12.985141 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:12.985411 (+ 270us) client_negotiation.cc:240] Negotiation successful
0812 05:25:12.985465 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":47,"thread_start_us":95,"threads_started":1}
I20260812 05:25:12.985567 26971 raft_consensus.cc:2454] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4de34691b57f4ca8a6e7760c2a3a8bde in term 0.
I20260812 05:25:12.985800 27050 leader_election.cc:302] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4de34691b57f4ca8a6e7760c2a3a8bde, 52dd10d3e5c4445f82eecd19bbcb39dd; no voters:
I20260812 05:25:12.985960 27200 raft_consensus.cc:2781] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:12.986012 27200 raft_consensus.cc:482] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:12.986040 27200 raft_consensus.cc:3037] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:12.986083 26822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08a1fa90c7cd44c9ba3e48633aff5a75" candidate_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "534fe3f7f4784a698217c859a828b21a" is_pre_election: true
I20260812 05:25:12.986315 26822 raft_consensus.cc:2454] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4de34691b57f4ca8a6e7760c2a3a8bde in term 0.
I20260812 05:25:12.987008 27200 raft_consensus.cc:504] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.987123 27200 leader_election.cc:288] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [CANDIDATE]: Term 1 election: Requested vote from peers 534fe3f7f4784a698217c859a828b21a (127.24.66.65:46009), 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699)
I20260812 05:25:12.987449 26822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08a1fa90c7cd44c9ba3e48633aff5a75" candidate_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "534fe3f7f4784a698217c859a828b21a"
I20260812 05:25:12.987496 26971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08a1fa90c7cd44c9ba3e48633aff5a75" candidate_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd"
I20260812 05:25:12.987538 26822 raft_consensus.cc:3037] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:12.987569 26971 raft_consensus.cc:3037] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:12.988392 26822 raft_consensus.cc:2454] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4de34691b57f4ca8a6e7760c2a3a8bde in term 1.
I20260812 05:25:12.988391 26971 raft_consensus.cc:2454] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4de34691b57f4ca8a6e7760c2a3a8bde in term 1.
I20260812 05:25:12.988618 27050 leader_election.cc:302] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4de34691b57f4ca8a6e7760c2a3a8bde, 534fe3f7f4784a698217c859a828b21a; no voters:
I20260812 05:25:12.988725 27200 raft_consensus.cc:2781] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:12.988844 27200 raft_consensus.cc:686] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 LEADER]: Becoming Leader. State: Replica: 4de34691b57f4ca8a6e7760c2a3a8bde, State: Running, Role: LEADER
I20260812 05:25:12.988937 27200 consensus_queue.cc:237] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:12.989717 26652 catalog_manager.cc:5495] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde reported cstate change: term changed from 0 to 1, leader changed from <none> to 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67). New cstate: current_term: 1 leader_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } health_report { overall_health: HEALTHY } } }
I20260812 05:25:13.012212 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:13.009162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43942 (local address 127.24.66.67:43307)
0812 05:25:13.009235 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:13.009237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:13.009244 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:25:13.009287 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:13.009288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:13.009349 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:13.009662 (+ 313us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:13.009828 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:13.010574 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:13.011397 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:13.011610 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:13.011682 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:13.011872 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:13.011991 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:13.012067 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:13.012085 (+ 18us) server_negotiation.cc:299] Negotiation successful
0812 05:25:13.012108 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 05:25:13.018194 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:13.015217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:41097 (local address 127.24.66.66:34699)
0812 05:25:13.015282 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:13.015284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:13.015488 (+ 204us) server_negotiation.cc:407] Connection header received
0812 05:25:13.015532 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:13.015533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:13.015542 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:13.015593 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:13.015834 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:13.016395 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:13.017429 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:13.017618 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:13.017678 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:13.017922 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:13.017951 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:13.017996 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:13.018106 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 05:25:13.018126 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:25:13.018379 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:13.015124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:34699 (local address 127.24.66.67:41097)
0812 05:25:13.015419 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:13.015430 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:13.015487 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:13.015611 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:13.015612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:13.015614 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:25:13.015751 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:13.015753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:13.016500 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:13.016501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:13.017345 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:13.017349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:13.017708 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:13.017709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:13.017819 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:13.017837 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:25:13.018037 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:25:13.018040 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:13.018104 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 05:25:13.018150 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":94,"threads_started":1}
I20260812 05:25:13.021045 26971 raft_consensus.cc:1261] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Refusing update from remote peer 4de34691b57f4ca8a6e7760c2a3a8bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:13.021068 26822 raft_consensus.cc:1261] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Refusing update from remote peer 4de34691b57f4ca8a6e7760c2a3a8bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:13.021718 27251 consensus_queue.cc:1035] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:13.021865 27200 consensus_queue.cc:1035] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:13.023613 27223 mvcc.cc:205] Tried to move back new op lower bound from 7317554434131910656 to 7317554434003279872. Current Snapshot: MvccSnapshot[applied={T|T < 7317554434131910656}]
I20260812 05:25:13.030164 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:13.027612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:60903 (local address 127.24.66.67:43307)
0812 05:25:13.027659 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:13.027661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:13.027690 (+ 29us) server_negotiation.cc:407] Connection header received
0812 05:25:13.027715 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:13.027716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:13.027726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:13.027776 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:13.027918 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:13.028451 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:13.029366 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:13.029601 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:13.029690 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:13.029872 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:13.029910 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:13.029990 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:13.030058 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 05:25:13.030080 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:25:13.030202 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:13.027547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:43307 (local address 127.24.66.66:60903)
0812 05:25:13.027599 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:13.027611 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:13.027654 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:13.027776 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:13.027777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:13.027778 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:25:13.027831 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:13.027832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:13.028546 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:13.028548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:13.029269 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:13.029271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:13.029701 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:13.029702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:13.029764 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:13.029780 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:25:13.029990 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:25:13.029993 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:13.030053 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 05:25:13.030110 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 05:25:13.030882 27225 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 05:25:14.553067 26653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:14.553195 26653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:14.554229 26653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a: Sending DeleteTablet for 3 replicas of tablet 08a1fa90c7cd44c9ba3e48633aff5a75
I20260812 05:25:14.554569 26801 tablet_service.cc:1514] Processing DeleteTablet for tablet 08a1fa90c7cd44c9ba3e48633aff5a75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:14 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:37942
I20260812 05:25:14.554629 27100 tablet_service.cc:1514] Processing DeleteTablet for tablet 08a1fa90c7cd44c9ba3e48633aff5a75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:14 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:43928
I20260812 05:25:14.554869 26950 tablet_service.cc:1514] Processing DeleteTablet for tablet 08a1fa90c7cd44c9ba3e48633aff5a75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:14 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:57566
I20260812 05:25:14.555423 27283 tablet_replica.cc:330] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a: stopping tablet replica
I20260812 05:25:14.555498 27283 raft_consensus.cc:2229] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:14.555543 27283 raft_consensus.cc:2258] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:14.555653 27285 tablet_replica.cc:330] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd: stopping tablet replica
I20260812 05:25:14.555711 27285 raft_consensus.cc:2229] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:14.555753 27285 raft_consensus.cc:2258] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:14.556025 27285 ts_tablet_manager.cc:1905] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:14.557727 27283 ts_tablet_manager.cc:1905] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:14.557930 27284 tablet_replica.cc:330] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde: stopping tablet replica
I20260812 05:25:14.558061 27284 raft_consensus.cc:2229] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:14.558168 27285 ts_tablet_manager.cc:1918] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 05:25:14.558209 27284 raft_consensus.cc:2258] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:14.558240 27285 log.cc:1196] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/08a1fa90c7cd44c9ba3e48633aff5a75
I20260812 05:25:14.558561 27285 ts_tablet_manager.cc:1939] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting consensus metadata
I20260812 05:25:14.558563 27284 ts_tablet_manager.cc:1905] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:14.559084 26638 catalog_manager.cc:4841] TS 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699): tablet 08a1fa90c7cd44c9ba3e48633aff5a75 (table test-table [id=1c1133e9b7a9487bbfea43ef1ddb1475]) successfully deleted
I20260812 05:25:14.559506 27283 ts_tablet_manager.cc:1918] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 05:25:14.559580 27283 log.cc:1196] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/wal/wals/08a1fa90c7cd44c9ba3e48633aff5a75
I20260812 05:25:14.559830 27283 ts_tablet_manager.cc:1939] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 534fe3f7f4784a698217c859a828b21a: Deleting consensus metadata
I20260812 05:25:14.560187 26638 catalog_manager.cc:4841] TS 534fe3f7f4784a698217c859a828b21a (127.24.66.65:46009): tablet 08a1fa90c7cd44c9ba3e48633aff5a75 (table test-table [id=1c1133e9b7a9487bbfea43ef1ddb1475]) successfully deleted
I20260812 05:25:14.560722 27284 ts_tablet_manager.cc:1918] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 05:25:14.560796 27284 log.cc:1196] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/08a1fa90c7cd44c9ba3e48633aff5a75
I20260812 05:25:14.561064 27284 ts_tablet_manager.cc:1939] T 08a1fa90c7cd44c9ba3e48633aff5a75 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting consensus metadata
I20260812 05:25:14.561694 26637 catalog_manager.cc:4841] TS 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67:43307): tablet 08a1fa90c7cd44c9ba3e48633aff5a75 (table test-table [id=1c1133e9b7a9487bbfea43ef1ddb1475]) successfully deleted
Aug 12 05:25:14 dist-test-slave-zpfg krb5kdc[26606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512314, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 05:25:14 dist-test-slave-zpfg krb5kdc[26606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512314, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:14.586452 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:14.577495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52084 (local address 127.24.66.126:40651)
0812 05:25:14.577711 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:14.577713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:14.577723 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:14.577835 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:14.577836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:14.577848 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:14.577924 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:14.578716 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:14.579361 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:14.580139 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:14.580439 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:14.580526 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:14.583173 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:14.583175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:14.583175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:14.583184 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:14.585007 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:14.585452 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:14.585453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:14.585454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:14.585526 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:14.585774 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:14.585775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:14.585776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:14.585918 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:14.586045 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:14.586199 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 05:25:14.586311 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":148,"threads_started":1}
I20260812 05:25:14.588178 26653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:14.588356 26653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:14.589962 26801 tablet_service.cc:1467] Processing CreateTablet for tablet aa668afb598b4125b7550516095aa184 (DEFAULT_TABLE table=test-table [id=73a218073c4a4f548adc9e66fa33df7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:14.589973 27100 tablet_service.cc:1467] Processing CreateTablet for tablet aa668afb598b4125b7550516095aa184 (DEFAULT_TABLE table=test-table [id=73a218073c4a4f548adc9e66fa33df7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:14.589974 26950 tablet_service.cc:1467] Processing CreateTablet for tablet aa668afb598b4125b7550516095aa184 (DEFAULT_TABLE table=test-table [id=73a218073c4a4f548adc9e66fa33df7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:14.590132 27100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa668afb598b4125b7550516095aa184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:14.590132 26950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa668afb598b4125b7550516095aa184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:14.590137 26801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa668afb598b4125b7550516095aa184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:14.591723 27296 tablet_bootstrap.cc:492] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde: Bootstrap starting.
I20260812 05:25:14.591877 27295 tablet_bootstrap.cc:492] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a: Bootstrap starting.
I20260812 05:25:14.591917 27297 tablet_bootstrap.cc:492] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd: Bootstrap starting.
I20260812 05:25:14.592310 27297 tablet_bootstrap.cc:654] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:14.592322 27295 tablet_bootstrap.cc:654] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:14.593075 27295 tablet_bootstrap.cc:492] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a: No bootstrap required, opened a new log
I20260812 05:25:14.593139 27295 ts_tablet_manager.cc:1397] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:14.593159 27297 tablet_bootstrap.cc:492] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd: No bootstrap required, opened a new log
I20260812 05:25:14.593175 27296 tablet_bootstrap.cc:654] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:14.593214 27297 ts_tablet_manager.cc:1397] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:14.593272 27295 raft_consensus.cc:348] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.593346 27295 raft_consensus.cc:374] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:14.593370 27295 raft_consensus.cc:729] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 534fe3f7f4784a698217c859a828b21a, State: Initialized, Role: FOLLOWER
I20260812 05:25:14.593343 27297 raft_consensus.cc:348] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.593421 27297 raft_consensus.cc:374] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:14.593454 27297 raft_consensus.cc:729] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dd10d3e5c4445f82eecd19bbcb39dd, State: Initialized, Role: FOLLOWER
I20260812 05:25:14.593442 27295 consensus_queue.cc:260] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.593525 27297 consensus_queue.cc:260] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.593649 27297 ts_tablet_manager.cc:1428] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:25:14.593648 27295 ts_tablet_manager.cc:1428] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:25:14.594125 27296 tablet_bootstrap.cc:492] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde: No bootstrap required, opened a new log
I20260812 05:25:14.594199 27296 ts_tablet_manager.cc:1397] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:25:14.594527 27296 raft_consensus.cc:348] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.594632 27296 raft_consensus.cc:374] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:14.594663 27296 raft_consensus.cc:729] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de34691b57f4ca8a6e7760c2a3a8bde, State: Initialized, Role: FOLLOWER
I20260812 05:25:14.594729 27296 consensus_queue.cc:260] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.594861 27296 ts_tablet_manager.cc:1428] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:14.632777 27301 raft_consensus.cc:482] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:14.632895 27301 raft_consensus.cc:504] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.633142 27301 leader_election.cc:288] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699), 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67:43307)
I20260812 05:25:14.633450 27121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa668afb598b4125b7550516095aa184" candidate_uuid: "534fe3f7f4784a698217c859a828b21a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" is_pre_election: true
I20260812 05:25:14.633450 26971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa668afb598b4125b7550516095aa184" candidate_uuid: "534fe3f7f4784a698217c859a828b21a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" is_pre_election: true
I20260812 05:25:14.633563 27121 raft_consensus.cc:2454] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 534fe3f7f4784a698217c859a828b21a in term 0.
I20260812 05:25:14.633563 26971 raft_consensus.cc:2454] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 534fe3f7f4784a698217c859a828b21a in term 0.
I20260812 05:25:14.633765 26748 leader_election.cc:302] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52dd10d3e5c4445f82eecd19bbcb39dd, 534fe3f7f4784a698217c859a828b21a; no voters:
I20260812 05:25:14.633924 27301 raft_consensus.cc:2781] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:14.633998 27301 raft_consensus.cc:482] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:14.634029 27301 raft_consensus.cc:3037] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:14.634944 27301 raft_consensus.cc:504] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.635310 27301 leader_election.cc:288] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [CANDIDATE]: Term 1 election: Requested vote from peers 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699), 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67:43307)
I20260812 05:25:14.635588 27121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa668afb598b4125b7550516095aa184" candidate_uuid: "534fe3f7f4784a698217c859a828b21a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde"
I20260812 05:25:14.635708 27121 raft_consensus.cc:3037] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:14.635761 26971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa668afb598b4125b7550516095aa184" candidate_uuid: "534fe3f7f4784a698217c859a828b21a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd"
I20260812 05:25:14.635869 26971 raft_consensus.cc:3037] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:14.636445 27121 raft_consensus.cc:2454] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 534fe3f7f4784a698217c859a828b21a in term 1.
I20260812 05:25:14.636662 26747 leader_election.cc:302] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4de34691b57f4ca8a6e7760c2a3a8bde, 534fe3f7f4784a698217c859a828b21a; no voters:
I20260812 05:25:14.636687 26971 raft_consensus.cc:2454] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 534fe3f7f4784a698217c859a828b21a in term 1.
I20260812 05:25:14.636772 27301 raft_consensus.cc:2781] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:14.636963 27301 raft_consensus.cc:686] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 1 LEADER]: Becoming Leader. State: Replica: 534fe3f7f4784a698217c859a828b21a, State: Running, Role: LEADER
I20260812 05:25:14.637032 27301 consensus_queue.cc:237] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.637672 26653 catalog_manager.cc:5495] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a reported cstate change: term changed from 0 to 1, leader changed from <none> to 534fe3f7f4784a698217c859a828b21a (127.24.66.65). New cstate: current_term: 1 leader_uuid: "534fe3f7f4784a698217c859a828b21a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } health_report { overall_health: UNKNOWN } } }
I20260812 05:25:14.657045 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:14.654491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37968 (local address 127.24.66.65:46009)
0812 05:25:14.654645 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:14.654646 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:14.654654 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:14.654673 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:14.654674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:14.654685 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:14.654749 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:14.654966 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:14.655566 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:14.656444 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:14.656640 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:14.656698 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:14.656827 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:14.656857 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:14.656895 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:14.656961 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 05:25:14.656980 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260812 05:25:14.659564 27121 raft_consensus.cc:1261] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Refusing update from remote peer 534fe3f7f4784a698217c859a828b21a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:14.659739 26971 raft_consensus.cc:1261] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Refusing update from remote peer 534fe3f7f4784a698217c859a828b21a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:14.659848 27302 consensus_queue.cc:1035] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:14.660080 27302 consensus_queue.cc:1035] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:14.670188 26653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:14.670332 26653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:14.671046 26653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a: Sending DeleteTablet for 3 replicas of tablet aa668afb598b4125b7550516095aa184
I20260812 05:25:14.671576 27100 tablet_service.cc:1514] Processing DeleteTablet for tablet aa668afb598b4125b7550516095aa184 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:14 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:43928
I20260812 05:25:14.671610 26950 tablet_service.cc:1514] Processing DeleteTablet for tablet aa668afb598b4125b7550516095aa184 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:14 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:57566
I20260812 05:25:14.671633 26801 tablet_service.cc:1514] Processing DeleteTablet for tablet aa668afb598b4125b7550516095aa184 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:14 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:37942
I20260812 05:25:14.671697 27284 tablet_replica.cc:330] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde: stopping tablet replica
I20260812 05:25:14.671736 27283 tablet_replica.cc:330] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a: stopping tablet replica
I20260812 05:25:14.671734 27285 tablet_replica.cc:330] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd: stopping tablet replica
I20260812 05:25:14.671758 27284 raft_consensus.cc:2229] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:14.671797 27284 raft_consensus.cc:2258] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:14.671798 27283 raft_consensus.cc:2229] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:14.671801 27285 raft_consensus.cc:2229] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:14.671861 27283 raft_consensus.cc:2258] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:14.671869 27285 raft_consensus.cc:2258] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:14.672180 27284 ts_tablet_manager.cc:1905] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:14.672214 27283 ts_tablet_manager.cc:1905] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:14.672394 27285 ts_tablet_manager.cc:1905] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:14.672339 26653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:14.672497 26653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:14.674227 27284 ts_tablet_manager.cc:1918] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:14.674249 27283 ts_tablet_manager.cc:1918] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:14.674317 27284 log.cc:1196] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/aa668afb598b4125b7550516095aa184
I20260812 05:25:14.674325 27283 log.cc:1196] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-0/wal/wals/aa668afb598b4125b7550516095aa184
I20260812 05:25:14.674602 27283 ts_tablet_manager.cc:1939] T aa668afb598b4125b7550516095aa184 P 534fe3f7f4784a698217c859a828b21a: Deleting consensus metadata
I20260812 05:25:14.674621 27285 ts_tablet_manager.cc:1918] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:14.674671 27285 log.cc:1196] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/aa668afb598b4125b7550516095aa184
I20260812 05:25:14.674643 27284 ts_tablet_manager.cc:1939] T aa668afb598b4125b7550516095aa184 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting consensus metadata
I20260812 05:25:14.674901 27285 ts_tablet_manager.cc:1939] T aa668afb598b4125b7550516095aa184 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting consensus metadata
I20260812 05:25:14.675022 26637 catalog_manager.cc:4841] TS 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67:43307): tablet aa668afb598b4125b7550516095aa184 (table test-table [id=73a218073c4a4f548adc9e66fa33df7d]) successfully deleted
I20260812 05:25:14.675204 26638 catalog_manager.cc:4841] TS 534fe3f7f4784a698217c859a828b21a (127.24.66.65:46009): tablet aa668afb598b4125b7550516095aa184 (table test-table [id=73a218073c4a4f548adc9e66fa33df7d]) successfully deleted
I20260812 05:25:14.675096 27100 tablet_service.cc:1467] Processing CreateTablet for tablet 346630631b154f108c5798044eb0b3c5 (DEFAULT_TABLE table=test-table [id=7dea70a6ebee424aa57fb2baa3d46abf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:14.675369 27100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346630631b154f108c5798044eb0b3c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:14.675411 26950 tablet_service.cc:1467] Processing CreateTablet for tablet 346630631b154f108c5798044eb0b3c5 (DEFAULT_TABLE table=test-table [id=7dea70a6ebee424aa57fb2baa3d46abf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:14.675526 26950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346630631b154f108c5798044eb0b3c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:14.676627 27297 tablet_bootstrap.cc:492] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd: Bootstrap starting.
I20260812 05:25:14.676649 26638 catalog_manager.cc:4841] TS 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699): tablet aa668afb598b4125b7550516095aa184 (table test-table [id=73a218073c4a4f548adc9e66fa33df7d]) successfully deleted
I20260812 05:25:14.676709 27296 tablet_bootstrap.cc:492] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde: Bootstrap starting.
I20260812 05:25:14.677070 27297 tablet_bootstrap.cc:654] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:14.677070 27296 tablet_bootstrap.cc:654] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:14.677395 26801 tablet_service.cc:1467] Processing CreateTablet for tablet 346630631b154f108c5798044eb0b3c5 (DEFAULT_TABLE table=test-table [id=7dea70a6ebee424aa57fb2baa3d46abf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:14.677523 26801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346630631b154f108c5798044eb0b3c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:14.677706 27297 tablet_bootstrap.cc:492] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd: No bootstrap required, opened a new log
I20260812 05:25:14.677762 27297 ts_tablet_manager.cc:1397] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:14.677897 27297 raft_consensus.cc:348] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.677977 27297 raft_consensus.cc:374] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:14.678000 27297 raft_consensus.cc:729] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dd10d3e5c4445f82eecd19bbcb39dd, State: Initialized, Role: FOLLOWER
I20260812 05:25:14.678046 27297 consensus_queue.cc:260] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.678154 27297 ts_tablet_manager.cc:1428] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:14.679039 27295 tablet_bootstrap.cc:492] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a: Bootstrap starting.
I20260812 05:25:14.679308 27296 tablet_bootstrap.cc:492] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde: No bootstrap required, opened a new log
I20260812 05:25:14.679394 27296 ts_tablet_manager.cc:1397] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 05:25:14.679474 27295 tablet_bootstrap.cc:654] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:14.679559 27296 raft_consensus.cc:348] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.679664 27296 raft_consensus.cc:374] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:14.679694 27296 raft_consensus.cc:729] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de34691b57f4ca8a6e7760c2a3a8bde, State: Initialized, Role: FOLLOWER
I20260812 05:25:14.679756 27296 consensus_queue.cc:260] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.679896 27296 ts_tablet_manager.cc:1428] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:14.680375 27295 tablet_bootstrap.cc:492] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a: No bootstrap required, opened a new log
I20260812 05:25:14.680419 27295 ts_tablet_manager.cc:1397] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:14.680594 27295 raft_consensus.cc:348] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.680689 27295 raft_consensus.cc:374] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:14.680706 27295 raft_consensus.cc:729] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 534fe3f7f4784a698217c859a828b21a, State: Initialized, Role: FOLLOWER
I20260812 05:25:14.680771 27295 consensus_queue.cc:260] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.680883 27295 ts_tablet_manager.cc:1428] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:14.731419 27251 raft_consensus.cc:482] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:14.731523 27251 raft_consensus.cc:504] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.731683 27251 leader_election.cc:288] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 534fe3f7f4784a698217c859a828b21a (127.24.66.65:46009), 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699)
I20260812 05:25:14.732067 26822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346630631b154f108c5798044eb0b3c5" candidate_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "534fe3f7f4784a698217c859a828b21a" is_pre_election: true
I20260812 05:25:14.732067 26971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346630631b154f108c5798044eb0b3c5" candidate_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" is_pre_election: true
I20260812 05:25:14.732196 26822 raft_consensus.cc:2454] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4de34691b57f4ca8a6e7760c2a3a8bde in term 0.
I20260812 05:25:14.732196 26971 raft_consensus.cc:2454] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4de34691b57f4ca8a6e7760c2a3a8bde in term 0.
I20260812 05:25:14.732411 27050 leader_election.cc:302] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4de34691b57f4ca8a6e7760c2a3a8bde, 52dd10d3e5c4445f82eecd19bbcb39dd; no voters:
I20260812 05:25:14.732548 27251 raft_consensus.cc:2781] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:14.732611 27251 raft_consensus.cc:482] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:14.732650 27251 raft_consensus.cc:3037] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:14.733498 27251 raft_consensus.cc:504] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.733666 27251 leader_election.cc:288] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [CANDIDATE]: Term 1 election: Requested vote from peers 534fe3f7f4784a698217c859a828b21a (127.24.66.65:46009), 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699)
I20260812 05:25:14.733863 26971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346630631b154f108c5798044eb0b3c5" candidate_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd"
I20260812 05:25:14.733868 26822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346630631b154f108c5798044eb0b3c5" candidate_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "534fe3f7f4784a698217c859a828b21a"
I20260812 05:25:14.733953 26971 raft_consensus.cc:3037] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:14.733953 26822 raft_consensus.cc:3037] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:14.734817 26971 raft_consensus.cc:2454] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4de34691b57f4ca8a6e7760c2a3a8bde in term 1.
I20260812 05:25:14.734905 26822 raft_consensus.cc:2454] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4de34691b57f4ca8a6e7760c2a3a8bde in term 1.
I20260812 05:25:14.734982 27050 leader_election.cc:302] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4de34691b57f4ca8a6e7760c2a3a8bde, 52dd10d3e5c4445f82eecd19bbcb39dd; no voters:
I20260812 05:25:14.735085 27251 raft_consensus.cc:2781] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:14.735169 27251 raft_consensus.cc:686] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 LEADER]: Becoming Leader. State: Replica: 4de34691b57f4ca8a6e7760c2a3a8bde, State: Running, Role: LEADER
I20260812 05:25:14.735230 27251 consensus_queue.cc:237] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } }
I20260812 05:25:14.735956 26653 catalog_manager.cc:5495] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde reported cstate change: term changed from 0 to 1, leader changed from <none> to 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67). New cstate: current_term: 1 leader_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4de34691b57f4ca8a6e7760c2a3a8bde" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43307 } health_report { overall_health: HEALTHY } } }
I20260812 05:25:14.757448 27316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:14.754870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.24.66.67:43307)
0812 05:25:14.755101 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:14.755102 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:14.755111 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:14.755131 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:14.755131 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:14.755181 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:14.755245 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:14.755434 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:14.755929 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:14.756841 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:14.757032 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:14.757086 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:14.757226 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:14.757257 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:14.757309 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:14.757365 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 05:25:14.757385 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":148,"threads_started":1}
I20260812 05:25:14.760031 26971 raft_consensus.cc:1261] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Refusing update from remote peer 4de34691b57f4ca8a6e7760c2a3a8bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:14.760070 26822 raft_consensus.cc:1261] T 346630631b154f108c5798044eb0b3c5 P 534fe3f7f4784a698217c859a828b21a [term 1 FOLLOWER]: Refusing update from remote peer 4de34691b57f4ca8a6e7760c2a3a8bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:14.760294 27274 consensus_queue.cc:1035] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "52dd10d3e5c4445f82eecd19bbcb39dd" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:14.760421 27251 consensus_queue.cc:1035] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "534fe3f7f4784a698217c859a828b21a" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 05:25:14.766230 27264 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 05:25:16.289767 26652 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:16.289903 26652 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:16.290966 26652 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83022e460b084ea29390b9597f34994a: Sending DeleteTablet for 3 replicas of tablet 346630631b154f108c5798044eb0b3c5
I20260812 05:25:16.291253 26801 tablet_service.cc:1514] Processing DeleteTablet for tablet 346630631b154f108c5798044eb0b3c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:16 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:37942
I20260812 05:25:16.291298 27100 tablet_service.cc:1514] Processing DeleteTablet for tablet 346630631b154f108c5798044eb0b3c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:16 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:43928
I20260812 05:25:16.291345 26949 tablet_service.cc:1514] Processing DeleteTablet for tablet 346630631b154f108c5798044eb0b3c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:16 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:57566
I20260812 05:25:16.291435 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 26729
I20260812 05:25:16.293785 27345 tablet_replica.cc:330] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde: stopping tablet replica
I20260812 05:25:16.293848 27345 raft_consensus.cc:2229] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:16.293897 27345 raft_consensus.cc:2258] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:16.294138 27345 ts_tablet_manager.cc:1905] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:16.295555 27347 tablet_replica.cc:330] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd: stopping tablet replica
I20260812 05:25:16.295621 27347 raft_consensus.cc:2229] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:16.295682 27347 raft_consensus.cc:2258] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:16.295943 27347 ts_tablet_manager.cc:1905] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:16.295989 27345 ts_tablet_manager.cc:1918] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 05:25:16.296047 27345 log.cc:1196] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/346630631b154f108c5798044eb0b3c5
I20260812 05:25:16.296322 27345 ts_tablet_manager.cc:1939] T 346630631b154f108c5798044eb0b3c5 P 4de34691b57f4ca8a6e7760c2a3a8bde: Deleting consensus metadata
I20260812 05:25:16.296708 26637 catalog_manager.cc:4841] TS 4de34691b57f4ca8a6e7760c2a3a8bde (127.24.66.67:43307): tablet 346630631b154f108c5798044eb0b3c5 (table test-table [id=7dea70a6ebee424aa57fb2baa3d46abf]) successfully deleted
I20260812 05:25:16.297434 27347 ts_tablet_manager.cc:1918] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 05:25:16.297516 27347 log.cc:1196] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/346630631b154f108c5798044eb0b3c5
I20260812 05:25:16.297724 27347 ts_tablet_manager.cc:1939] T 346630631b154f108c5798044eb0b3c5 P 52dd10d3e5c4445f82eecd19bbcb39dd: Deleting consensus metadata
I20260812 05:25:16.298125 26638 catalog_manager.cc:4841] TS 52dd10d3e5c4445f82eecd19bbcb39dd (127.24.66.66:34699): tablet 346630631b154f108c5798044eb0b3c5 (table test-table [id=7dea70a6ebee424aa57fb2baa3d46abf]) successfully deleted
W20260812 05:25:16.300423 26638 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:46009 (error 108)
I20260812 05:25:16.300719 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 26883
I20260812 05:25:16.301892 27349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:16.300860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:46009 (local address 127.0.0.1:37976)
0812 05:25:16.301788 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:16.301842 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:46009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":900,"thread_start_us":57,"threads_started":1}
W20260812 05:25:16.301978 26638 catalog_manager.cc:4584] TS 534fe3f7f4784a698217c859a828b21a (127.24.66.65:46009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 346630631b154f108c5798044eb0b3c5: Network error: Client connection negotiation failed: client connection to 127.24.66.65:46009: connect: Connection refused (error 111)
I20260812 05:25:16.310420 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 27032
I20260812 05:25:16.319959 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 26620
2026-08-12T05:25:16Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5972 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:16 dist-test-slave-zpfg krb5kdc[27352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:16 dist-test-slave-zpfg krb5kdc[27352](info): set up 2 sockets
Aug 12 05:25:16 dist-test-slave-zpfg krb5kdc[27352](info): commencing operation
krb5kdc: starting...
W20260812 05:25:17.496225 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:17 dist-test-slave-zpfg krb5kdc[27352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:25:17.635070 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40039
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:40039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:17.721447 27366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:17.721709 27366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:17.721750 27366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:17.723052 27366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:17.723127 27366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:17.723150 27366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:17.723171 27366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:17.723203 27366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:17.724732 27366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:40039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:17.725044 27366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:17.725299 27366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:17.730756 27373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:17.730767 27375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:17.730768 27372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:17.731120 27366 server_base.cc:1034] running on GCE node
I20260812 05:25:17.731386 27366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:17.731722 27366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:17.732914 27366 hybrid_clock.cc:648] HybridClock initialized: now 1786512317732906 us; error 42 us; skew 500 ppm
Aug 12 05:25:17 dist-test-slave-zpfg krb5kdc[27352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512317, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:17.739934 27366 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:17.740415 27366 webserver.cc:460] Webserver started at http://127.24.66.126:45187/ using document root <none> and password file <none>
I20260812 05:25:17.740705 27366 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:17.740778 27366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:17.740952 27366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:17.742153 27366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "c2965a5eea2b4033899d3f588bb6d92e"
format_stamp: "Formatted at 2026-08-12 05:25:17 on dist-test-slave-zpfg"
server_key: "d0ff856cde45d2af85a7ae0f5851fc33"
server_key_iv: "056bd3e35ac5ce134ced9db7733c7fda"
server_key_version: "encryptionkey@0"
I20260812 05:25:17.742615 27366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "c2965a5eea2b4033899d3f588bb6d92e"
format_stamp: "Formatted at 2026-08-12 05:25:17 on dist-test-slave-zpfg"
server_key: "d0ff856cde45d2af85a7ae0f5851fc33"
server_key_iv: "056bd3e35ac5ce134ced9db7733c7fda"
server_key_version: "encryptionkey@0"
I20260812 05:25:17.744174 27366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:17.745074 27382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:17.745437 27366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:17.745522 27366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "c2965a5eea2b4033899d3f588bb6d92e"
format_stamp: "Formatted at 2026-08-12 05:25:17 on dist-test-slave-zpfg"
server_key: "d0ff856cde45d2af85a7ae0f5851fc33"
server_key_iv: "056bd3e35ac5ce134ced9db7733c7fda"
server_key_version: "encryptionkey@0"
I20260812 05:25:17.745620 27366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:17.772401 27366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:17.775278 27366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:17.775468 27366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:17.780300 27366 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:40039
I20260812 05:25:17.780339 27438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:40039 every 8 connection(s)
I20260812 05:25:17.780726 27366 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:17.781704 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:17.781993 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 27366
I20260812 05:25:17.782099 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:17.782409 24841 external_mini_cluster.cc:1428] Setting key fad5af46f46ff885af8d8425727bd619
I20260812 05:25:17.785003 27439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e: Bootstrap starting.
I20260812 05:25:17.785641 27439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:17.785950 27439 log.cc:824] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:17.786720 27439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e: No bootstrap required, opened a new log
I20260812 05:25:17.788975 27439 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2965a5eea2b4033899d3f588bb6d92e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40039 } }
I20260812 05:25:17.789147 27439 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:17.789207 27439 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2965a5eea2b4033899d3f588bb6d92e, State: Initialized, Role: FOLLOWER
I20260812 05:25:17.789399 27439 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2965a5eea2b4033899d3f588bb6d92e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40039 } }
I20260812 05:25:17.789500 27439 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 12 05:25:17 dist-test-slave-zpfg krb5kdc[27352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:17.789546 27439 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:17.789598 27439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:17.790439 27439 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2965a5eea2b4033899d3f588bb6d92e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40039 } }
I20260812 05:25:17.790562 27439 leader_election.cc:302] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2965a5eea2b4033899d3f588bb6d92e; no voters:
I20260812 05:25:17.790745 27439 leader_election.cc:288] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:17.790835 27445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:17.791039 27439 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:17.791406 27445 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [term 1 LEADER]: Becoming Leader. State: Replica: c2965a5eea2b4033899d3f588bb6d92e, State: Running, Role: LEADER
I20260812 05:25:17.791877 27445 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2965a5eea2b4033899d3f588bb6d92e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40039 } }
I20260812 05:25:17.792470 27446 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2965a5eea2b4033899d3f588bb6d92e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2965a5eea2b4033899d3f588bb6d92e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40039 } } }
I20260812 05:25:17.792552 27446 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:17.792785 27447 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2965a5eea2b4033899d3f588bb6d92e. Latest consensus state: current_term: 1 leader_uuid: "c2965a5eea2b4033899d3f588bb6d92e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2965a5eea2b4033899d3f588bb6d92e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40039 } } }
I20260812 05:25:17.792961 27447 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e [sys.catalog]: This master's current role is: LEADER
W20260812 05:25:17.793185 27460 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:25:17.793252 27460 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:25:17.793674 27461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:17.793866 27461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:17.794747 27443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:17.783028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51208 (local address 127.24.66.126:40039)
0812 05:25:17.783236 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:17.783244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:17.783272 (+ 28us) server_negotiation.cc:407] Connection header received
0812 05:25:17.783676 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:17.783683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:17.783731 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:17.783866 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:17.784715 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:17.785717 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:17.786674 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:17.787243 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:17.787376 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:17.790192 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:17.790197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:17.790197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:17.790211 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:17.792510 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:17.793038 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:17.793041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:17.793041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:17.793155 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:17.793396 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:17.793397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:17.793397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:17.793732 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:17.793906 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:17.794399 (+ 493us) server_negotiation.cc:299] Negotiation successful
0812 05:25:17.794552 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260812 05:25:17.796439 27461 catalog_manager.cc:1347] Generated new cluster ID: 0d2417a59bb145029e5cbf587f52c318
I20260812 05:25:17.796484 27461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:17.807339 27461 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:17.808146 27461 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:17.816789 27461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2965a5eea2b4033899d3f588bb6d92e: Generated new TSK 0
I20260812 05:25:17.817068 27461 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:25:17.857896 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40039
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:17.945876 27467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:17.946115 27467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:17.946141 27467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:17.947451 27467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:17.947500 27467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:17.947530 27467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:17.949116 27467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:17.949553 27467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:17.949849 27467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:17.955282 27476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:17.955307 27474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:17.955514 27473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:17.955713 27467 server_base.cc:1034] running on GCE node
I20260812 05:25:17.955928 27467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:17.956195 27467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:17.957372 27467 hybrid_clock.cc:648] HybridClock initialized: now 1786512317957370 us; error 43 us; skew 500 ppm
Aug 12 05:25:17 dist-test-slave-zpfg krb5kdc[27352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512317, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:17.960139 27467 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:25:17.960687 27467 webserver.cc:460] Webserver started at http://127.24.66.65:33235/ using document root <none> and password file <none>
I20260812 05:25:17.960965 27467 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:17.961038 27467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:17.961206 27467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:17.962301 27467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "b7ef20c4354045c9bc411e2bf8c15e2c"
format_stamp: "Formatted at 2026-08-12 05:25:17 on dist-test-slave-zpfg"
server_key: "2edf2434b8fa1d7f85ba1fcc4649173c"
server_key_iv: "361e317b4d522fa24e98db717369001b"
server_key_version: "encryptionkey@0"
I20260812 05:25:17.962769 27467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "b7ef20c4354045c9bc411e2bf8c15e2c"
format_stamp: "Formatted at 2026-08-12 05:25:17 on dist-test-slave-zpfg"
server_key: "2edf2434b8fa1d7f85ba1fcc4649173c"
server_key_iv: "361e317b4d522fa24e98db717369001b"
server_key_version: "encryptionkey@0"
I20260812 05:25:17.964389 27467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:17.965281 27483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:17.965484 27467 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:17.965593 27467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "b7ef20c4354045c9bc411e2bf8c15e2c"
format_stamp: "Formatted at 2026-08-12 05:25:17 on dist-test-slave-zpfg"
server_key: "2edf2434b8fa1d7f85ba1fcc4649173c"
server_key_iv: "361e317b4d522fa24e98db717369001b"
server_key_version: "encryptionkey@0"
I20260812 05:25:17.965669 27467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:17.982188 27467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:17.984731 27467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:17.985002 27467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:17.985270 27467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:17.985664 27467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:17.985733 27467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:17.985776 27467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:17.985805 27467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:17.992628 27467 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:46537
I20260812 05:25:17.992740 27603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:46537 every 8 connection(s)
I20260812 05:25:17.993120 27467 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:17.994133 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 27467
I20260812 05:25:17.994246 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:17.994479 24841 external_mini_cluster.cc:1428] Setting key 04f50e1e92d03755af9035e66c633d16
Aug 12 05:25:17 dist-test-slave-zpfg krb5kdc[27352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512317, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
I20260812 05:25:18.005638 27443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:17.993985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:36377 (local address 127.24.66.126:40039)
0812 05:25:17.994044 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:17.994047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:17.994381 (+ 334us) server_negotiation.cc:407] Connection header received
0812 05:25:17.995329 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:17.995330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:17.995343 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:17.995387 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:17.996553 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:17.997117 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:17.998294 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:17.998617 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:17.998715 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:18.001805 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:18.001807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:18.001808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:18.001819 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:18.003857 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:18.004399 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:18.004401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:18.004402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:18.004475 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:18.004794 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:18.004795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:18.004795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:18.004936 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:18.005025 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:18.005383 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 05:25:18.005516 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:25:18.005993 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:17.993906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40039 (local address 127.24.66.65:36377)
0812 05:25:17.994180 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:17.994253 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:17.995060 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:17.995504 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:17.995508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:17.996189 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:17.996412 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:17.996418 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:17.997230 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:17.997231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:17.998181 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:17.998184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:17.998732 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:17.998733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:17.998890 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:17.999530 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:17.999539 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:18.001676 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:18.003970 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:18.003984 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:18.003987 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:18.004297 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:18.004560 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:18.004561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:18.004561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:18.004706 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:18.005014 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:18.005024 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:18.005207 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 05:25:18.005353 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 05:25:18.005564 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":151,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
I20260812 05:25:18.006713 27604 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40039
I20260812 05:25:18.006840 27604 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:18.007066 27604 heartbeater.cc:511] Master 127.24.66.126:40039 requested a full tablet report, sending...
I20260812 05:25:18.007767 27401 ts_manager.cc:194] Registered new tserver with Master: b7ef20c4354045c9bc411e2bf8c15e2c (127.24.66.65:46537)
I20260812 05:25:18.008816 27401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:36377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:25:18.026449 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40039
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:18.118319 27609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:18.118592 27609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:18.118618 27609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:18.119796 27609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:18.119849 27609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:18.119877 27609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:18.121330 27609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:18.121542 27609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:18.121739 27609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:18.127439 27617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:18.127483 27615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:18.127435 27614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:18.127810 27609 server_base.cc:1034] running on GCE node
I20260812 05:25:18.128105 27609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:18.128407 27609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:18.129666 27609 hybrid_clock.cc:648] HybridClock initialized: now 1786512318129567 us; error 35 us; skew 500 ppm
Aug 12 05:25:18 dist-test-slave-zpfg krb5kdc[27352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512318, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:18.132790 27609 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:25:18.133442 27609 webserver.cc:460] Webserver started at http://127.24.66.66:38219/ using document root <none> and password file <none>
I20260812 05:25:18.133749 27609 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:18.133831 27609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:18.134006 27609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:18.135350 27609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "f3364c8eed194812a3ee1e7e0ebbcb3e"
format_stamp: "Formatted at 2026-08-12 05:25:18 on dist-test-slave-zpfg"
server_key: "ddca2252f9a60aaa80591619ffffff33"
server_key_iv: "fa0a04e71550bf56885329ef3ed341e0"
server_key_version: "encryptionkey@0"
I20260812 05:25:18.135783 27609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "f3364c8eed194812a3ee1e7e0ebbcb3e"
format_stamp: "Formatted at 2026-08-12 05:25:18 on dist-test-slave-zpfg"
server_key: "ddca2252f9a60aaa80591619ffffff33"
server_key_iv: "fa0a04e71550bf56885329ef3ed341e0"
server_key_version: "encryptionkey@0"
I20260812 05:25:18.137336 27609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:18.138223 27624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:18.138439 27609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:18.138559 27609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "f3364c8eed194812a3ee1e7e0ebbcb3e"
format_stamp: "Formatted at 2026-08-12 05:25:18 on dist-test-slave-zpfg"
server_key: "ddca2252f9a60aaa80591619ffffff33"
server_key_iv: "fa0a04e71550bf56885329ef3ed341e0"
server_key_version: "encryptionkey@0"
I20260812 05:25:18.138656 27609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:18.150314 27609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:18.152716 27609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:18.152899 27609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:18.153194 27609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:18.153550 27609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:18.153609 27609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:18.153645 27609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:18.153672 27609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:18.160320 27609 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:37405
I20260812 05:25:18.160363 27741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:37405 every 8 connection(s)
I20260812 05:25:18.160706 27609 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:18.163189 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 27609
I20260812 05:25:18.163316 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:18.163497 24841 external_mini_cluster.cc:1428] Setting key f7e00878d38c2080aa733c33d5d5d519
Aug 12 05:25:18 dist-test-slave-zpfg krb5kdc[27352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512318, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:18.172010 27443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:18.161526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:42811 (local address 127.24.66.126:40039)
0812 05:25:18.161597 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:18.161600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:18.161853 (+ 253us) server_negotiation.cc:407] Connection header received
0812 05:25:18.162470 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:18.162472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:18.162482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:18.162539 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:18.163630 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:18.164363 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:18.165445 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:18.165756 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:18.165845 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:18.168629 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:18.168630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:18.168631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:18.168640 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:18.170295 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:18.170926 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:18.170928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:18.170928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:18.170988 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:18.171299 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:18.171299 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:18.171299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:18.171439 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:18.171492 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:18.171856 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 05:25:18.171940 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:25:18.172302 27744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:18.161516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40039 (local address 127.24.66.66:42811)
0812 05:25:18.161724 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:18.161761 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:18.162308 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:18.162611 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:18.162615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:18.163254 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:18.163522 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:18.163528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:18.164473 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:18.164474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:18.165348 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:18.165351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:18.165858 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:18.165859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:18.165969 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:18.166657 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:18.166669 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:18.168508 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:18.170446 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:18.170455 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:18.170458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:18.170844 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:18.171063 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:18.171064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:18.171064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:18.171225 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:18.171508 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:18.171514 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:18.171712 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 05:25:18.171838 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 05:25:18.172029 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260812 05:25:18.172935 27742 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40039
I20260812 05:25:18.173055 27742 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:18.173278 27742 heartbeater.cc:511] Master 127.24.66.126:40039 requested a full tablet report, sending...
I20260812 05:25:18.173722 27401 ts_manager.cc:194] Registered new tserver with Master: f3364c8eed194812a3ee1e7e0ebbcb3e (127.24.66.66:37405)
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
I20260812 05:25:18.174166 27401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:42811
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:25:18.197381 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40039
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:18.288996 27747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:18.289479 27747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:18.289531 27747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:18.292068 27747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:18.292166 27747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:18.292232 27747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:18.295730 27747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:18.296224 27747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:18.296550 27747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:18.306242 27757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:18.306237 27753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:18.306315 27755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:18.306694 27747 server_base.cc:1034] running on GCE node
I20260812 05:25:18.307293 27747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:18.307825 27747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:18.309067 27747 hybrid_clock.cc:648] HybridClock initialized: now 1786512318309024 us; error 53 us; skew 500 ppm
Aug 12 05:25:18 dist-test-slave-zpfg krb5kdc[27352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512318, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:18.314441 27747 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:25:18.315446 27747 webserver.cc:460] Webserver started at http://127.24.66.67:35875/ using document root <none> and password file <none>
I20260812 05:25:18.315919 27747 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:18.316054 27747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:18.316308 27747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:18.318610 27747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "920fdaf284de4203b7eb58d6fc9360df"
format_stamp: "Formatted at 2026-08-12 05:25:18 on dist-test-slave-zpfg"
server_key: "f91155b6efbbd75035b926bf007e7e5d"
server_key_iv: "2028079a5d4bbe66377b5230260a8548"
server_key_version: "encryptionkey@0"
I20260812 05:25:18.319327 27747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "920fdaf284de4203b7eb58d6fc9360df"
format_stamp: "Formatted at 2026-08-12 05:25:18 on dist-test-slave-zpfg"
server_key: "f91155b6efbbd75035b926bf007e7e5d"
server_key_iv: "2028079a5d4bbe66377b5230260a8548"
server_key_version: "encryptionkey@0"
I20260812 05:25:18.322567 27747 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 05:25:18.324921 27764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:18.325402 27747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:18.325716 27747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "920fdaf284de4203b7eb58d6fc9360df"
format_stamp: "Formatted at 2026-08-12 05:25:18 on dist-test-slave-zpfg"
server_key: "f91155b6efbbd75035b926bf007e7e5d"
server_key_iv: "2028079a5d4bbe66377b5230260a8548"
server_key_version: "encryptionkey@0"
I20260812 05:25:18.325984 27747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:18.349534 27747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:18.354176 27747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:18.354547 27747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:18.355021 27747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:18.355618 27747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:18.355670 27747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:18.355717 27747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:18.355744 27747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:18.370110 27747 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:46065
I20260812 05:25:18.370241 27885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:46065 every 8 connection(s)
I20260812 05:25:18.370962 27747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:18.375075 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 27747
I20260812 05:25:18.375181 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:18.375540 24841 external_mini_cluster.cc:1428] Setting key d33b7f9cc591fd7a1f930c952a545477
Aug 12 05:25:18 dist-test-slave-zpfg krb5kdc[27352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512318, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:18.390040 27443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:18.372221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:34917 (local address 127.24.66.126:40039)
0812 05:25:18.372353 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:18.372356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:18.372916 (+ 560us) server_negotiation.cc:407] Connection header received
0812 05:25:18.373794 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:18.373795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:18.373826 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:18.373944 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:18.375992 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:18.377091 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:18.379312 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:18.379659 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:18.379826 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:18.384010 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:18.384012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:18.384012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:18.384028 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:18.386760 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:18.387861 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:18.387870 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:18.387870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:18.388008 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:18.388632 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:18.388633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:18.388633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:18.388840 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:18.389016 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:18.389642 (+ 626us) server_negotiation.cc:299] Negotiation successful
0812 05:25:18.389819 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 05:25:18.390607 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:18.372171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40039 (local address 127.24.66.67:34917)
0812 05:25:18.372588 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:18.372727 (+ 139us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:18.373655 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:18.374033 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:18.374037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:18.375330 (+ 1293us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:18.375808 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:18.375825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:18.377359 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:18.377360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:18.379078 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:18.379081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:18.379851 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:18.379852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:18.380023 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:18.381095 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:18.381114 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:18.383838 (+ 2724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:18.386947 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:18.386962 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:18.386971 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:18.387514 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:18.388154 (+ 640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:18.388155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:18.388156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:18.388419 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:18.388997 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:18.389013 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:18.389310 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 05:25:18.389547 (+ 237us) client_negotiation.cc:240] Negotiation successful
0812 05:25:18.389962 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":202,"threads_started":1}
I20260812 05:25:18.391692 27887 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40039
I20260812 05:25:18.391937 27887 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:18.392349 27887 heartbeater.cc:511] Master 127.24.66.126:40039 requested a full tablet report, sending...
I20260812 05:25:18.393085 27401 ts_manager.cc:194] Registered new tserver with Master: 920fdaf284de4203b7eb58d6fc9360df (127.24.66.67:46065)
I20260812 05:25:18.393759 27401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:34917
I20260812 05:25:18.400434 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 05:25:18.426442 27443 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:18.424216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.24.66.126:40039)
0812 05:25:18.424381 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:18.424385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:18.424612 (+ 227us) server_negotiation.cc:407] Connection header received
0812 05:25:18.424823 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:18.424824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:18.424856 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:18.425004 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:18.426247 (+ 1243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51218: BlockingRecv error: recv got EOF from 127.0.0.1:51218 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 05:25:18.428439 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 27467
I20260812 05:25:18.442814 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 27609
I20260812 05:25:18.454988 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 27747
I20260812 05:25:18.468359 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 27366
2026-08-12T05:25:18Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2159 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:18 dist-test-slave-zpfg krb5kdc[27899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:18 dist-test-slave-zpfg krb5kdc[27899](info): set up 2 sockets
Aug 12 05:25:18 dist-test-slave-zpfg krb5kdc[27899](info): commencing operation
krb5kdc: starting...
W20260812 05:25:19.824036 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.001s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:19 dist-test-slave-zpfg krb5kdc[27899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:25:19.962638 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:43105
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:42123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:43105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:20.051262 27912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:20.051534 27912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:20.051574 27912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:20.052788 27912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:20.052867 27912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:20.052896 27912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:20.052927 27912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:20.052976 27912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:20.054656 27912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:42123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:43105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:43105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:20.054949 27912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:20.055197 27912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:20.061249 27918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:20.061234 27919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:20.061208 27912 server_base.cc:1034] running on GCE node
W20260812 05:25:20.061272 27921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:20.061856 27912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:20.062251 27912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:20.063453 27912 hybrid_clock.cc:648] HybridClock initialized: now 1786512320063432 us; error 30 us; skew 500 ppm
Aug 12 05:25:20 dist-test-slave-zpfg krb5kdc[27899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512320, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:20.066622 27912 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:20.067145 27912 webserver.cc:460] Webserver started at http://127.24.66.126:37861/ using document root <none> and password file <none>
I20260812 05:25:20.067418 27912 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:20.067497 27912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:20.067663 27912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:20.068867 27912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "3140af3865bf40649a5fd45cd81a9a55"
format_stamp: "Formatted at 2026-08-12 05:25:20 on dist-test-slave-zpfg"
server_key: "6138285ef57d2ff3293cfa9c94dbd415"
server_key_iv: "4f5d61cb9a07ca66d95aac5fed677a00"
server_key_version: "encryptionkey@0"
I20260812 05:25:20.069308 27912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "3140af3865bf40649a5fd45cd81a9a55"
format_stamp: "Formatted at 2026-08-12 05:25:20 on dist-test-slave-zpfg"
server_key: "6138285ef57d2ff3293cfa9c94dbd415"
server_key_iv: "4f5d61cb9a07ca66d95aac5fed677a00"
server_key_version: "encryptionkey@0"
I20260812 05:25:20.070871 27912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:20.071828 27928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:20.072063 27912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:20.072183 27912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "3140af3865bf40649a5fd45cd81a9a55"
format_stamp: "Formatted at 2026-08-12 05:25:20 on dist-test-slave-zpfg"
server_key: "6138285ef57d2ff3293cfa9c94dbd415"
server_key_iv: "4f5d61cb9a07ca66d95aac5fed677a00"
server_key_version: "encryptionkey@0"
I20260812 05:25:20.072273 27912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:20.085129 27912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:20.087421 27912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:20.087598 27912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:20.092195 27912 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:43105
I20260812 05:25:20.092327 27983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:43105 every 8 connection(s)
I20260812 05:25:20.092808 27912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:20.093513 27984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:20.096020 27984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55: Bootstrap starting.
I20260812 05:25:20.096616 27984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:20.096920 27984 log.cc:824] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:20.097700 27984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55: No bootstrap required, opened a new log
I20260812 05:25:20.098605 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 27912
I20260812 05:25:20.098704 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:20.098879 24841 external_mini_cluster.cc:1428] Setting key 4b120274df5705d90316d0b6bef1fe3f
I20260812 05:25:20.100559 27984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140af3865bf40649a5fd45cd81a9a55" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43105 } }
I20260812 05:25:20.100703 27984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:20.100724 27984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3140af3865bf40649a5fd45cd81a9a55, State: Initialized, Role: FOLLOWER
I20260812 05:25:20.100901 27984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140af3865bf40649a5fd45cd81a9a55" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43105 } }
I20260812 05:25:20.100987 27984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:20.101012 27984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:20.101081 27984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:20.101723 27984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140af3865bf40649a5fd45cd81a9a55" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43105 } }
I20260812 05:25:20.101857 27984 leader_election.cc:302] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3140af3865bf40649a5fd45cd81a9a55; no voters:
I20260812 05:25:20.102065 27984 leader_election.cc:288] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:20.102100 27990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:20.102218 27990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [term 1 LEADER]: Becoming Leader. State: Replica: 3140af3865bf40649a5fd45cd81a9a55, State: Running, Role: LEADER
I20260812 05:25:20.102305 27990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140af3865bf40649a5fd45cd81a9a55" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43105 } }
I20260812 05:25:20.102378 27984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:20.103536 27992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3140af3865bf40649a5fd45cd81a9a55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140af3865bf40649a5fd45cd81a9a55" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43105 } } }
I20260812 05:25:20.103643 27992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:20.104040 27991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3140af3865bf40649a5fd45cd81a9a55. Latest consensus state: current_term: 1 leader_uuid: "3140af3865bf40649a5fd45cd81a9a55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3140af3865bf40649a5fd45cd81a9a55" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43105 } } }
I20260812 05:25:20.104053 28005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:20.104445 27991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:20.104939 28005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:20.106652 28005 catalog_manager.cc:1347] Generated new cluster ID: 849687774fd2428fb67d3d43cebe90be
I20260812 05:25:20.106709 28005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:25:20 dist-test-slave-zpfg krb5kdc[27899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:20.111337 27989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:20.099513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46022 (local address 127.24.66.126:43105)
0812 05:25:20.099712 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:20.099721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:20.099749 (+ 28us) server_negotiation.cc:407] Connection header received
0812 05:25:20.100560 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:20.100568 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:20.100635 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:20.100836 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:20.101628 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:20.102944 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:20.104259 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:20.104861 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:20.104978 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:20.107784 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:20.107789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:20.107789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:20.107801 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:20.109664 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:20.110059 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:20.110062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:20.110062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:20.110189 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:20.110465 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:20.110466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:20.110466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:20.110727 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:20.110930 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:20.111001 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 05:25:20.111165 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260812 05:25:20.112526 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:25:20.132413 27989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:20.127340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46034 (local address 127.24.66.126:43105)
0812 05:25:20.127394 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:20.127396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:20.127568 (+ 172us) server_negotiation.cc:407] Connection header received
0812 05:25:20.127634 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:20.127635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:20.127650 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:20.127707 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:20.128232 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:20.129276 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:20.130483 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:20.130844 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:20.130919 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:20.131798 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:20.131799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:20.131800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:20.132082 (+ 282us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:20.132115 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:20.132123 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:20.132192 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:20.132272 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 05:25:20.132308 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 05:25:20.137840 28005 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:20.138689 28005 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:20.142370 28005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3140af3865bf40649a5fd45cd81a9a55: Generated new TSK 0
I20260812 05:25:20.142589 28005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:25:20.153635 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 27912
2026-08-12T05:25:20Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1671 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:20 dist-test-slave-zpfg krb5kdc[28019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:20 dist-test-slave-zpfg krb5kdc[28019](info): set up 2 sockets
Aug 12 05:25:20 dist-test-slave-zpfg krb5kdc[28019](info): commencing operation
krb5kdc: starting...
W20260812 05:25:21.327845 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:21 dist-test-slave-zpfg krb5kdc[28019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:25:21.470626 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:38267
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:38267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:21.558436 28033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:21.558697 28033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:21.558735 28033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:21.559940 28033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:21.560016 28033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:21.560040 28033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:21.560061 28033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:21.560093 28033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:21.561774 28033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:38267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:38267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:21.562075 28033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:21.562367 28033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:21.567832 28039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:21.567840 28040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:21.568006 28033 server_base.cc:1034] running on GCE node
W20260812 05:25:21.567811 28042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:21.568447 28033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:21.568748 28033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:21.569896 28033 hybrid_clock.cc:648] HybridClock initialized: now 1786512321569863 us; error 48 us; skew 500 ppm
Aug 12 05:25:21 dist-test-slave-zpfg krb5kdc[28019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512321, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:21.572786 28033 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:21.573227 28033 webserver.cc:460] Webserver started at http://127.24.66.126:46813/ using document root <none> and password file <none>
I20260812 05:25:21.573504 28033 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:21.573581 28033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:21.573735 28033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:21.575003 28033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "40c0d1dc27b1461c8aebbb8ed9f3f7c6"
format_stamp: "Formatted at 2026-08-12 05:25:21 on dist-test-slave-zpfg"
server_key: "5803a2a24dcc78679b8c82a2517e9e00"
server_key_iv: "85021dec9e15d8be8f91a538e432d825"
server_key_version: "encryptionkey@0"
I20260812 05:25:21.575410 28033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "40c0d1dc27b1461c8aebbb8ed9f3f7c6"
format_stamp: "Formatted at 2026-08-12 05:25:21 on dist-test-slave-zpfg"
server_key: "5803a2a24dcc78679b8c82a2517e9e00"
server_key_iv: "85021dec9e15d8be8f91a538e432d825"
server_key_version: "encryptionkey@0"
I20260812 05:25:21.577046 28033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 05:25:21.577883 28049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:21.578078 28033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:21.578195 28033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "40c0d1dc27b1461c8aebbb8ed9f3f7c6"
format_stamp: "Formatted at 2026-08-12 05:25:21 on dist-test-slave-zpfg"
server_key: "5803a2a24dcc78679b8c82a2517e9e00"
server_key_iv: "85021dec9e15d8be8f91a538e432d825"
server_key_version: "encryptionkey@0"
I20260812 05:25:21.578284 28033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:21.602276 28033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:21.604671 28033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:21.604866 28033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:21.609529 28033 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:38267
I20260812 05:25:21.609571 28104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:38267 every 8 connection(s)
I20260812 05:25:21.609959 28033 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:21.611130 28105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:21.613576 28105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6: Bootstrap starting.
I20260812 05:25:21.614208 28105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:21.614563 28105 log.cc:824] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:21.615371 28105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6: No bootstrap required, opened a new log
I20260812 05:25:21.617328 28105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c0d1dc27b1461c8aebbb8ed9f3f7c6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38267 } }
I20260812 05:25:21.617481 28105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:21.617504 28105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c0d1dc27b1461c8aebbb8ed9f3f7c6, State: Initialized, Role: FOLLOWER
I20260812 05:25:21.617650 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 28033
I20260812 05:25:21.617672 28105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c0d1dc27b1461c8aebbb8ed9f3f7c6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38267 } }
I20260812 05:25:21.617771 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:21.617789 28105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:21.617832 28105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:21.617890 28105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:21.618021 24841 external_mini_cluster.cc:1428] Setting key 7229888867e6524db1a6a8887b54b42a
I20260812 05:25:21.619313 28105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c0d1dc27b1461c8aebbb8ed9f3f7c6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38267 } }
I20260812 05:25:21.619436 28105 leader_election.cc:302] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40c0d1dc27b1461c8aebbb8ed9f3f7c6; no voters:
I20260812 05:25:21.619602 28105 leader_election.cc:288] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:21.619634 28111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:21.619833 28111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [term 1 LEADER]: Becoming Leader. State: Replica: 40c0d1dc27b1461c8aebbb8ed9f3f7c6, State: Running, Role: LEADER
I20260812 05:25:21.619894 28105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:21.619946 28111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c0d1dc27b1461c8aebbb8ed9f3f7c6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38267 } }
I20260812 05:25:21.620386 28113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40c0d1dc27b1461c8aebbb8ed9f3f7c6. Latest consensus state: current_term: 1 leader_uuid: "40c0d1dc27b1461c8aebbb8ed9f3f7c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c0d1dc27b1461c8aebbb8ed9f3f7c6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38267 } } }
I20260812 05:25:21.620571 28113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:21.620865 28120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:21.621541 28112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40c0d1dc27b1461c8aebbb8ed9f3f7c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c0d1dc27b1461c8aebbb8ed9f3f7c6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38267 } } }
I20260812 05:25:21.621654 28112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:21.621817 28120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:21.623646 28120 catalog_manager.cc:1347] Generated new cluster ID: e9d0f7abf91641c8bc93a89ba6fe20e1
I20260812 05:25:21.623703 28120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:25:21 dist-test-slave-zpfg krb5kdc[28019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:21.629208 28120 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:21.629748 28120 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:21.631515 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:21.618696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54972 (local address 127.24.66.126:38267)
0812 05:25:21.618968 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:21.618979 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:21.619021 (+ 42us) server_negotiation.cc:407] Connection header received
0812 05:25:21.619144 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:21.619152 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:21.619210 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:21.619384 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:21.620208 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:21.621863 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:21.622984 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:21.623582 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:21.623685 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:21.626778 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:21.626783 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:21.626783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:21.626797 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:21.629058 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:21.629542 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:21.629547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:21.629548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:21.629790 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:21.630029 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:21.630030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:21.630031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:21.630335 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:21.630529 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:21.631121 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 05:25:21.631292 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":137,"threads_started":1}
I20260812 05:25:21.642846 28120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40c0d1dc27b1461c8aebbb8ed9f3f7c6: Generated new TSK 0
I20260812 05:25:21.643078 28120 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:25:21.678404 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:38267
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:21.788726 28135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:21.788970 28135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:21.788997 28135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:21.790144 28135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:21.790184 28135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:21.790211 28135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:21.791745 28135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:38267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:21.791957 28135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:21.792145 28135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:21.797465 28144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:21.797581 28135 server_base.cc:1034] running on GCE node
W20260812 05:25:21.797474 28142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:21.797477 28141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:21.797996 28135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:21.798245 28135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:21.799410 28135 hybrid_clock.cc:648] HybridClock initialized: now 1786512321799396 us; error 34 us; skew 500 ppm
Aug 12 05:25:21 dist-test-slave-zpfg krb5kdc[28019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512321, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:21.802230 28135 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:25:21.802734 28135 webserver.cc:460] Webserver started at http://127.24.66.65:41785/ using document root <none> and password file <none>
I20260812 05:25:21.802995 28135 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:21.803043 28135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:21.803196 28135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:21.804335 28135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "498fbf202330424baf905dea8a31d0ed"
format_stamp: "Formatted at 2026-08-12 05:25:21 on dist-test-slave-zpfg"
server_key: "a6ffd64697f883ed1636ea629696cba2"
server_key_iv: "93343d18d75a145173f3d4a832b33b99"
server_key_version: "encryptionkey@0"
I20260812 05:25:21.804726 28135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "498fbf202330424baf905dea8a31d0ed"
format_stamp: "Formatted at 2026-08-12 05:25:21 on dist-test-slave-zpfg"
server_key: "a6ffd64697f883ed1636ea629696cba2"
server_key_iv: "93343d18d75a145173f3d4a832b33b99"
server_key_version: "encryptionkey@0"
I20260812 05:25:21.806164 28135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:21.807009 28151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:21.807200 28135 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:21.807278 28135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "498fbf202330424baf905dea8a31d0ed"
format_stamp: "Formatted at 2026-08-12 05:25:21 on dist-test-slave-zpfg"
server_key: "a6ffd64697f883ed1636ea629696cba2"
server_key_iv: "93343d18d75a145173f3d4a832b33b99"
server_key_version: "encryptionkey@0"
I20260812 05:25:21.807359 28135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:21.817467 28135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:21.819595 28135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:21.819785 28135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:21.820029 28135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:21.820370 28135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:21.820434 28135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:21.820479 28135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:21.820528 28135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:21.827054 28135 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:33519
I20260812 05:25:21.827100 28271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:33519 every 8 connection(s)
I20260812 05:25:21.827468 28135 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
Aug 12 05:25:21 dist-test-slave-zpfg krb5kdc[28019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512321, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:21.835502 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 28135
I20260812 05:25:21.835647 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:21.835870 24841 external_mini_cluster.cc:1428] Setting key 8cd5fc6cbdd2a9c73c1cc048bcbce188
I20260812 05:25:21.838069 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:21.828362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:58395 (local address 127.24.66.126:38267)
0812 05:25:21.828423 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:21.828425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:21.828634 (+ 209us) server_negotiation.cc:407] Connection header received
0812 05:25:21.829141 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:21.829142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:21.829153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:21.829211 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:21.830227 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:21.830815 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:21.831930 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:21.832133 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:21.832194 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:21.834652 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:21.834654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:21.834654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:21.834661 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:21.836303 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:21.836856 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:21.836857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:21.836857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:21.836933 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:21.837267 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:21.837268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:21.837268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:21.837426 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:21.837502 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:21.837815 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 05:25:21.837925 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 05:25:21.838099 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:21.828353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38267 (local address 127.24.66.65:58395)
0812 05:25:21.828500 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:21.828538 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:21.829049 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:21.829266 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:21.829269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:21.829928 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:21.830135 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:21.830141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:21.830924 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:21.830925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:21.831824 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:21.831827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:21.832253 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:21.832254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:21.832414 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:21.833031 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:21.833040 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:21.834556 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:21.836431 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:21.836440 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:21.836442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:21.836758 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:21.837007 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:21.837008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:21.837008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:21.837134 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:21.837519 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:21.837527 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:21.837667 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 05:25:21.837764 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 05:25:21.837915 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260812 05:25:21.838686 28272 heartbeater.cc:347] Connected to a master server at 127.24.66.126:38267
I20260812 05:25:21.838827 28272 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:21.839026 28272 heartbeater.cc:511] Master 127.24.66.126:38267 requested a full tablet report, sending...
I20260812 05:25:21.839598 28066 ts_manager.cc:194] Registered new tserver with Master: 498fbf202330424baf905dea8a31d0ed (127.24.66.65:33519)
I20260812 05:25:21.840224 28066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:58395
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:25:21.866933 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:38267
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:21.952257 28277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:21.952515 28277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:21.952543 28277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:21.953758 28277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:21.953819 28277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:21.953866 28277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:21.955421 28277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:38267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:21.955696 28277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:21.955955 28277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:21.961431 28282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:21.961580 28283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:21.961658 28286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:21.961993 28277 server_base.cc:1034] running on GCE node
I20260812 05:25:21.962201 28277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:21.962543 28277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:21.963733 28277 hybrid_clock.cc:648] HybridClock initialized: now 1786512321963750 us; error 72 us; skew 500 ppm
Aug 12 05:25:21 dist-test-slave-zpfg krb5kdc[28019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512321, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:21.966825 28277 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:25:21.967326 28277 webserver.cc:460] Webserver started at http://127.24.66.66:38663/ using document root <none> and password file <none>
I20260812 05:25:21.967590 28277 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:21.967638 28277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:21.967777 28277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:21.968941 28277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "47665efb159b4caa9474f55fb02e15a4"
format_stamp: "Formatted at 2026-08-12 05:25:21 on dist-test-slave-zpfg"
server_key: "58f324958c87722afa3ff6d1a46eacc1"
server_key_iv: "0186756fe7c0c92b4952f7ade7168dc3"
server_key_version: "encryptionkey@0"
I20260812 05:25:21.969370 28277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "47665efb159b4caa9474f55fb02e15a4"
format_stamp: "Formatted at 2026-08-12 05:25:21 on dist-test-slave-zpfg"
server_key: "58f324958c87722afa3ff6d1a46eacc1"
server_key_iv: "0186756fe7c0c92b4952f7ade7168dc3"
server_key_version: "encryptionkey@0"
I20260812 05:25:21.970863 28277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:21.971740 28293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:21.971954 28277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:21.972064 28277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "47665efb159b4caa9474f55fb02e15a4"
format_stamp: "Formatted at 2026-08-12 05:25:21 on dist-test-slave-zpfg"
server_key: "58f324958c87722afa3ff6d1a46eacc1"
server_key_iv: "0186756fe7c0c92b4952f7ade7168dc3"
server_key_version: "encryptionkey@0"
I20260812 05:25:21.972146 28277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:21.997442 28277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:22.000214 28277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:22.000415 28277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:22.000684 28277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:22.001482 28277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:22.001569 28277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:22.001610 28277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:22.001641 28277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:22.009352 28277 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:46049
I20260812 05:25:22.009626 28414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:46049 every 8 connection(s)
I20260812 05:25:22.009840 28277 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:22.013574 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 28277
I20260812 05:25:22.013724 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:22.013929 24841 external_mini_cluster.cc:1428] Setting key 72d90ebfa6ad5800d015dcfb8e4486eb
Aug 12 05:25:22 dist-test-slave-zpfg krb5kdc[28019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512321, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:22.023073 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:22.010914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:47387 (local address 127.24.66.126:38267)
0812 05:25:22.010965 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:22.010968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:22.011162 (+ 194us) server_negotiation.cc:407] Connection header received
0812 05:25:22.011782 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:22.011784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:22.011795 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:22.011863 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:22.012923 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:22.013615 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:22.015209 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:22.015507 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:22.015593 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:22.018710 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:22.018712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:22.018712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:22.018720 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:22.020843 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:22.021809 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:22.021810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:22.021811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:22.021884 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:22.022233 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:22.022234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:22.022234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:22.022450 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:22.022516 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:22.022918 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 05:25:22.023002 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:25:22.023559 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:22.010818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38267 (local address 127.24.66.66:47387)
0812 05:25:22.010997 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:22.011044 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:22.011670 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:22.011919 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:22.011923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:22.012555 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:22.012781 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:22.012787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:22.013875 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:22.013877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:22.015098 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:22.015102 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:22.015630 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:22.015631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:22.015766 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:22.016504 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:22.016514 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:22.018566 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:22.020955 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:22.020968 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:22.020971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:22.021713 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:22.021975 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:22.021976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:22.021977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:22.022154 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:22.022522 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:22.022531 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:22.022751 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 05:25:22.022899 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 05:25:22.023110 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260812 05:25:22.024169 28415 heartbeater.cc:347] Connected to a master server at 127.24.66.126:38267
I20260812 05:25:22.024294 28415 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:22.024529 28415 heartbeater.cc:511] Master 127.24.66.126:38267 requested a full tablet report, sending...
I20260812 05:25:22.025077 28066 ts_manager.cc:194] Registered new tserver with Master: 47665efb159b4caa9474f55fb02e15a4 (127.24.66.66:46049)
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
I20260812 05:25:22.025537 28066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:47387
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:25:22.047041 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:38267
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:22.135221 28420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:22.135473 28420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:22.135499 28420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:22.136763 28420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:22.136819 28420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:22.136849 28420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:22.138496 28420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:38267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:22.138764 28420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:22.139025 28420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:22.144706 28425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:22.144855 28428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:22.144742 28426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:22.144977 28420 server_base.cc:1034] running on GCE node
I20260812 05:25:22.145326 28420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:22.145627 28420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:22.146813 28420 hybrid_clock.cc:648] HybridClock initialized: now 1786512322146783 us; error 39 us; skew 500 ppm
Aug 12 05:25:22 dist-test-slave-zpfg krb5kdc[28019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512322, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:22.149961 28420 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:25:22.150585 28420 webserver.cc:460] Webserver started at http://127.24.66.67:44863/ using document root <none> and password file <none>
I20260812 05:25:22.150890 28420 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:22.150974 28420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:22.151149 28420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:22.152375 28420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "231d211ed92246939feef556b478d9f1"
format_stamp: "Formatted at 2026-08-12 05:25:22 on dist-test-slave-zpfg"
server_key: "fb29b192bada2b97e24205d796c119d9"
server_key_iv: "a9f865d0d52933bbc1212725d4b1b86b"
server_key_version: "encryptionkey@0"
I20260812 05:25:22.152819 28420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "231d211ed92246939feef556b478d9f1"
format_stamp: "Formatted at 2026-08-12 05:25:22 on dist-test-slave-zpfg"
server_key: "fb29b192bada2b97e24205d796c119d9"
server_key_iv: "a9f865d0d52933bbc1212725d4b1b86b"
server_key_version: "encryptionkey@0"
I20260812 05:25:22.154493 28420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 05:25:22.155493 28435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:22.155815 28420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:22.155934 28420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "231d211ed92246939feef556b478d9f1"
format_stamp: "Formatted at 2026-08-12 05:25:22 on dist-test-slave-zpfg"
server_key: "fb29b192bada2b97e24205d796c119d9"
server_key_iv: "a9f865d0d52933bbc1212725d4b1b86b"
server_key_version: "encryptionkey@0"
I20260812 05:25:22.156014 28420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:22.172292 28420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:22.174863 28420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:22.175045 28420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:22.175390 28420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:22.175740 28420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:22.175796 28420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:22.175829 28420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:22.175858 28420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:22.182271 28420 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:41335
I20260812 05:25:22.182380 28553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:41335 every 8 connection(s)
I20260812 05:25:22.182752 28420 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:22.183852 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 28420
I20260812 05:25:22.183967 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:22.184312 24841 external_mini_cluster.cc:1428] Setting key d1039bb890f001bdc8682ffdbceb33f3
Aug 12 05:25:22 dist-test-slave-zpfg krb5kdc[28019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512322, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:22.193878 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:22.183599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:58759 (local address 127.24.66.126:38267)
0812 05:25:22.183686 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:22.183689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:22.183942 (+ 253us) server_negotiation.cc:407] Connection header received
0812 05:25:22.184710 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:22.184711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:22.184740 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:22.184807 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:22.186016 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:22.186611 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:22.187709 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:22.187905 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:22.187963 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:22.190566 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:22.190568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:22.190568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:22.190576 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:22.192372 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:22.192835 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:22.192836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:22.192836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:22.192899 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:22.193233 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:22.193234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:22.193234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:22.193383 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:22.193452 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:22.193727 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 05:25:22.193800 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 05:25:22.194213 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:22.183549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38267 (local address 127.24.66.67:58759)
0812 05:25:22.183771 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:22.183826 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:22.184522 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:22.184897 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:22.184901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:22.185714 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:22.185926 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:22.185931 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:22.186696 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:22.186698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:22.187574 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:22.187576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:22.187987 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:22.187988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:22.188093 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:22.188712 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:22.188722 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:22.190412 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:22.192475 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:22.192484 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:22.192486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:22.192735 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:22.193031 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:22.193032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:22.193032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:22.193159 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:22.193455 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:22.193462 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:22.193602 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 05:25:22.193699 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 05:25:22.193937 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":121,"threads_started":1}
I20260812 05:25:22.194840 28554 heartbeater.cc:347] Connected to a master server at 127.24.66.126:38267
I20260812 05:25:22.194969 28554 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:22.195204 28554 heartbeater.cc:511] Master 127.24.66.126:38267 requested a full tablet report, sending...
I20260812 05:25:22.195751 28066 ts_manager.cc:194] Registered new tserver with Master: 231d211ed92246939feef556b478d9f1 (127.24.66.67:41335)
I20260812 05:25:22.196163 28066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:58759
I20260812 05:25:22.196166 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 05:25:22 dist-test-slave-zpfg krb5kdc[28019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512322, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 05:25:22 dist-test-slave-zpfg krb5kdc[28019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512322, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:22.222328 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:22.214009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.24.66.126:38267)
0812 05:25:22.214092 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:22.214095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:22.214183 (+ 88us) server_negotiation.cc:407] Connection header received
0812 05:25:22.214280 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:22.214281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:22.214293 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:22.214409 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:22.215013 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:22.215515 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:22.216383 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:22.216583 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:22.216656 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:22.219129 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:22.219130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:22.219131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:22.219139 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:22.220818 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:22.221360 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:22.221362 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:22.221363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:22.221433 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:22.221663 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:22.221664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:22.221664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:22.221855 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:22.221978 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:22.222114 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 05:25:22.222211 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 05:25:22.222649 28066 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54974
I20260812 05:25:22.223220 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 28135
I20260812 05:25:22.230168 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 28277
I20260812 05:25:22.240306 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 28420
I20260812 05:25:22.246455 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 28033
2026-08-12T05:25:22Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2094 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:22 dist-test-slave-zpfg krb5kdc[28566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:22 dist-test-slave-zpfg krb5kdc[28566](info): set up 2 sockets
Aug 12 05:25:22 dist-test-slave-zpfg krb5kdc[28566](info): commencing operation
krb5kdc: starting...
W20260812 05:25:23.343705 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:23 dist-test-slave-zpfg krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:25:23.482502 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40211
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:35729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:40211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:23.568332 28579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:23.568562 28579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:23.568585 28579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:23.569802 28579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:23.569849 28579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:23.569864 28579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:23.569903 28579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:23.569927 28579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:23.571533 28579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:40211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:23.571826 28579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:23.572080 28579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:23.577731 28584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:23.577709 28587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:23.577803 28585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:23.577952 28579 server_base.cc:1034] running on GCE node
I20260812 05:25:23.578537 28579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:23.578909 28579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:23.580124 28579 hybrid_clock.cc:648] HybridClock initialized: now 1786512323580052 us; error 118 us; skew 500 ppm
Aug 12 05:25:23 dist-test-slave-zpfg krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512323, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:23.588692 28579 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:23.589218 28579 webserver.cc:460] Webserver started at http://127.24.66.126:38083/ using document root <none> and password file <none>
I20260812 05:25:23.589458 28579 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:23.589509 28579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:23.589625 28579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:23.590930 28579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "ec14ca2c0d53457ca8ef1850c4773bb0"
format_stamp: "Formatted at 2026-08-12 05:25:23 on dist-test-slave-zpfg"
server_key: "c313a731dcb5687c94c216599989b670"
server_key_iv: "0c8390401b0d43ce942a834993823a3b"
server_key_version: "encryptionkey@0"
I20260812 05:25:23.591293 28579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "ec14ca2c0d53457ca8ef1850c4773bb0"
format_stamp: "Formatted at 2026-08-12 05:25:23 on dist-test-slave-zpfg"
server_key: "c313a731dcb5687c94c216599989b670"
server_key_iv: "0c8390401b0d43ce942a834993823a3b"
server_key_version: "encryptionkey@0"
I20260812 05:25:23.592851 28579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:25:23.593695 28594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:23.593935 28579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:23.594036 28579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "ec14ca2c0d53457ca8ef1850c4773bb0"
format_stamp: "Formatted at 2026-08-12 05:25:23 on dist-test-slave-zpfg"
server_key: "c313a731dcb5687c94c216599989b670"
server_key_iv: "0c8390401b0d43ce942a834993823a3b"
server_key_version: "encryptionkey@0"
I20260812 05:25:23.594100 28579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:23.615779 28579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:23.618321 28579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:23.618511 28579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:23.622849 28579 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:40211
I20260812 05:25:23.622908 28651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:40211 every 8 connection(s)
I20260812 05:25:23.623452 28579 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:23.624185 28652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:23.626698 28652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0: Bootstrap starting.
I20260812 05:25:23.627345 28652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:23.627765 28652 log.cc:824] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:23.628813 28652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0: No bootstrap required, opened a new log
I20260812 05:25:23.628973 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 28579
I20260812 05:25:23.629097 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:23.629308 24841 external_mini_cluster.cc:1428] Setting key e9398d1bf69f4256bee83c73b3a39c5a
I20260812 05:25:23.631851 28652 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec14ca2c0d53457ca8ef1850c4773bb0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40211 } }
I20260812 05:25:23.632057 28652 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:23.632100 28652 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec14ca2c0d53457ca8ef1850c4773bb0, State: Initialized, Role: FOLLOWER
I20260812 05:25:23.632295 28652 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec14ca2c0d53457ca8ef1850c4773bb0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40211 } }
I20260812 05:25:23.632390 28652 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:23.632416 28652 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:23.632476 28652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:23.633184 28652 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec14ca2c0d53457ca8ef1850c4773bb0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40211 } }
I20260812 05:25:23.633294 28652 leader_election.cc:302] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec14ca2c0d53457ca8ef1850c4773bb0; no voters:
I20260812 05:25:23.633488 28652 leader_election.cc:288] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:23.633584 28657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:23.633739 28657 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [term 1 LEADER]: Becoming Leader. State: Replica: ec14ca2c0d53457ca8ef1850c4773bb0, State: Running, Role: LEADER
I20260812 05:25:23.633795 28652 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:23.634501 28657 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec14ca2c0d53457ca8ef1850c4773bb0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40211 } }
I20260812 05:25:23.635730 28659 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec14ca2c0d53457ca8ef1850c4773bb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec14ca2c0d53457ca8ef1850c4773bb0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40211 } } }
I20260812 05:25:23.635859 28659 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:23.635943 28658 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec14ca2c0d53457ca8ef1850c4773bb0. Latest consensus state: current_term: 1 leader_uuid: "ec14ca2c0d53457ca8ef1850c4773bb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec14ca2c0d53457ca8ef1850c4773bb0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40211 } } }
I20260812 05:25:23.636059 28658 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0 [sys.catalog]: This master's current role is: LEADER
W20260812 05:25:23.636153 28673 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:25:23.636211 28673 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:25:23.636289 28675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:23.636524 28675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:23.638449 28675 catalog_manager.cc:1347] Generated new cluster ID: 3e9fc87908a64fc1b96be395f646886d
I20260812 05:25:23.638505 28675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:25:23 dist-test-slave-zpfg krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:23.643406 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:23.629985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39276 (local address 127.24.66.126:40211)
0812 05:25:23.630221 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:23.630232 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:23.630272 (+ 40us) server_negotiation.cc:407] Connection header received
0812 05:25:23.631793 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:23.631811 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:23.631884 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:23.632056 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:23.632966 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:23.634265 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:23.635363 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:23.636164 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:23.636298 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:23.639483 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:23.639487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:23.639488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:23.639502 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:23.641551 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:23.641984 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:23.641988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:23.641989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:23.642119 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:23.642377 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:23.642378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:23.642378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:23.642674 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:23.642872 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:23.643012 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 05:25:23.643173 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":125,"threads_started":1}
I20260812 05:25:23.660655 28675 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:23.661257 28675 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:23.667899 28675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec14ca2c0d53457ca8ef1850c4773bb0: Generated new TSK 0
I20260812 05:25:23.668129 28675 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:25:23.706019 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40211
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:35729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:23.795480 28680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:23.795728 28680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:23.795758 28680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:23.797248 28680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:23.797305 28680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:23.797456 28680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:23.799983 28680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:23.800294 28680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:23.800611 28680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:23.806354 28686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:23.806322 28688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:23.806558 28685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:23.806770 28680 server_base.cc:1034] running on GCE node
I20260812 05:25:23.806999 28680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:23.807281 28680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:23.808436 28680 hybrid_clock.cc:648] HybridClock initialized: now 1786512323808415 us; error 39 us; skew 500 ppm
Aug 12 05:25:23 dist-test-slave-zpfg krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512323, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:23.811237 28680 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:25:23.811740 28680 webserver.cc:460] Webserver started at http://127.24.66.65:44103/ using document root <none> and password file <none>
I20260812 05:25:23.811987 28680 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:23.812060 28680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:23.812218 28680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:23.813285 28680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "2369498a8ae24ae1ab69da06e7c67575"
format_stamp: "Formatted at 2026-08-12 05:25:23 on dist-test-slave-zpfg"
server_key: "ef5a1c1843162f24a1c20e40316b9ecf"
server_key_iv: "4f990e4acb778c2841461ab53b8bc338"
server_key_version: "encryptionkey@0"
I20260812 05:25:23.813657 28680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "2369498a8ae24ae1ab69da06e7c67575"
format_stamp: "Formatted at 2026-08-12 05:25:23 on dist-test-slave-zpfg"
server_key: "ef5a1c1843162f24a1c20e40316b9ecf"
server_key_iv: "4f990e4acb778c2841461ab53b8bc338"
server_key_version: "encryptionkey@0"
I20260812 05:25:23.815285 28680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:23.816138 28695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:23.816323 28680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:23.816432 28680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "2369498a8ae24ae1ab69da06e7c67575"
format_stamp: "Formatted at 2026-08-12 05:25:23 on dist-test-slave-zpfg"
server_key: "ef5a1c1843162f24a1c20e40316b9ecf"
server_key_iv: "4f990e4acb778c2841461ab53b8bc338"
server_key_version: "encryptionkey@0"
I20260812 05:25:23.816511 28680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:23.828162 28680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:23.830749 28680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:23.830945 28680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:23.831220 28680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:23.831557 28680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:23.831614 28680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:23.831650 28680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:23.831694 28680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:23.838496 28680 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:33731
I20260812 05:25:23.838604 28814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:33731 every 8 connection(s)
I20260812 05:25:23.839030 28680 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:23.842092 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 28680
I20260812 05:25:23.842201 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:23.842486 24841 external_mini_cluster.cc:1428] Setting key c5703632693c050e8be8246a1b41b4e5
Aug 12 05:25:23 dist-test-slave-zpfg krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512323, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:23.851034 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:23.839965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:37055 (local address 127.24.66.126:40211)
0812 05:25:23.840015 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:23.840017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:23.840250 (+ 233us) server_negotiation.cc:407] Connection header received
0812 05:25:23.841224 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:23.841225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:23.841237 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:23.841304 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:23.841921 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:23.842818 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:23.844207 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:23.844469 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:23.844551 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:23.847090 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:23.847092 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:23.847093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:23.847102 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:23.849202 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:23.849703 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:23.849705 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:23.849706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:23.849781 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:23.850125 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:23.850125 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:23.850126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:23.850299 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:23.850399 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:23.850781 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 05:25:23.850910 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:25:23.851125 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:23.839940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40211 (local address 127.24.66.65:37055)
0812 05:25:23.840111 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:23.840149 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:23.841104 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:23.841365 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:23.841368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:23.841630 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:23.841836 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:23.841841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:23.842932 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:23.842933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:23.844081 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:23.844084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:23.844586 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:23.844588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:23.844731 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:23.845383 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:23.845393 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:23.846977 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:23.849312 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:23.849321 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:23.849323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:23.849609 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:23.849899 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:23.849900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:23.849901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:23.850035 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:23.850426 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:23.850434 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:23.850604 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 05:25:23.850734 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 05:25:23.850895 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
I20260812 05:25:23.851807 28816 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40211
I20260812 05:25:23.851919 28816 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:23.852144 28816 heartbeater.cc:511] Master 127.24.66.126:40211 requested a full tablet report, sending...
I20260812 05:25:23.852837 28614 ts_manager.cc:194] Registered new tserver with Master: 2369498a8ae24ae1ab69da06e7c67575 (127.24.66.65:33731)
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
I20260812 05:25:23.853675 28614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:37055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:25:23.871611 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40211
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:35729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:23.956766 28821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:23.957032 28821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:23.957074 28821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:23.958380 28821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:23.958463 28821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:23.958509 28821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:23.960080 28821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:23.960373 28821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:23.960624 28821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:23.966387 28828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:23.966401 28821 server_base.cc:1034] running on GCE node
W20260812 05:25:23.966408 28831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:23.966403 28829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:23.966974 28821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:23.967262 28821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:23.968458 28821 hybrid_clock.cc:648] HybridClock initialized: now 1786512323968442 us; error 28 us; skew 500 ppm
Aug 12 05:25:23 dist-test-slave-zpfg krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512323, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:23.971280 28821 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:25:23.971858 28821 webserver.cc:460] Webserver started at http://127.24.66.66:33579/ using document root <none> and password file <none>
I20260812 05:25:23.972107 28821 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:23.972182 28821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:23.972352 28821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:23.973496 28821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "0b029c0970db4d04b3db6d5129e91620"
format_stamp: "Formatted at 2026-08-12 05:25:23 on dist-test-slave-zpfg"
server_key: "2261842c34f2063a17e33d055b4e9fbe"
server_key_iv: "f1c037fc15d0b98022d37e78e1f2f8ea"
server_key_version: "encryptionkey@0"
I20260812 05:25:23.973893 28821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "0b029c0970db4d04b3db6d5129e91620"
format_stamp: "Formatted at 2026-08-12 05:25:23 on dist-test-slave-zpfg"
server_key: "2261842c34f2063a17e33d055b4e9fbe"
server_key_iv: "f1c037fc15d0b98022d37e78e1f2f8ea"
server_key_version: "encryptionkey@0"
I20260812 05:25:23.975512 28821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:23.976363 28838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:23.976544 28821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:23.976655 28821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "0b029c0970db4d04b3db6d5129e91620"
format_stamp: "Formatted at 2026-08-12 05:25:23 on dist-test-slave-zpfg"
server_key: "2261842c34f2063a17e33d055b4e9fbe"
server_key_iv: "f1c037fc15d0b98022d37e78e1f2f8ea"
server_key_version: "encryptionkey@0"
I20260812 05:25:23.976732 28821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:23.987679 28821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:23.990260 28821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:23.990499 28821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:23.990764 28821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:23.991107 28821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:23.991164 28821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:23.991200 28821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:23.991261 28821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:23.998066 28821 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:42901
I20260812 05:25:23.998122 28963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:42901 every 8 connection(s)
I20260812 05:25:23.998581 28821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:25:24 dist-test-slave-zpfg krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512323, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:24.007880 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 28821
I20260812 05:25:24.008010 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:24.008224 24841 external_mini_cluster.cc:1428] Setting key 084bae061ed82c103dc9172f7164b594
I20260812 05:25:24.010142 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:23.999566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:44255 (local address 127.24.66.126:40211)
0812 05:25:23.999638 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:23.999640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:23.999843 (+ 203us) server_negotiation.cc:407] Connection header received
0812 05:25:24.000889 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:24.000891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:24.000903 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:24.000985 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:24.001666 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.002454 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.003575 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.003808 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.003877 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:24.006230 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:24.006232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:24.006232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:24.006240 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:24.008173 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:24.008685 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:24.008686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:24.008687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:24.008744 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:24.009107 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:24.009108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:24.009108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:24.009251 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:24.009330 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:24.009939 (+ 609us) server_negotiation.cc:299] Negotiation successful
0812 05:25:24.010037 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 05:25:24.010524 28967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:23.999535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40211 (local address 127.24.66.66:44255)
0812 05:25:23.999702 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:23.999747 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:24.000736 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:24.001048 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:24.001052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:24.001341 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:24.001548 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:24.001553 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.002574 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.002575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:24.003468 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:24.003470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.003899 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.003900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:24.004012 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:24.004623 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:24.004638 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:24.006121 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:24.008283 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:24.008293 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:24.008297 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:24.008573 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:24.008815 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:24.008816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:24.008817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:24.009016 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:24.009331 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:24.009340 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:24.009495 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 05:25:24.009911 (+ 416us) client_negotiation.cc:240] Negotiation successful
0812 05:25:24.010110 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260812 05:25:24.011073 28964 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40211
I20260812 05:25:24.011171 28964 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:24.011384 28964 heartbeater.cc:511] Master 127.24.66.126:40211 requested a full tablet report, sending...
I20260812 05:25:24.011802 28614 ts_manager.cc:194] Registered new tserver with Master: 0b029c0970db4d04b3db6d5129e91620 (127.24.66.66:42901)
I20260812 05:25:24.012356 28614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:44255
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:25:24.038210 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40211
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:35729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:24.123916 28970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:24.124136 28970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:24.124161 28970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:24.125427 28970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:24.125478 28970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:24.125505 28970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:24.127054 28970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:24.127266 28970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:24.127465 28970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:24.132968 28979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:24.132968 28976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:24.132968 28977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:24.133277 28970 server_base.cc:1034] running on GCE node
I20260812 05:25:24.133543 28970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:24.133838 28970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:24.135068 28970 hybrid_clock.cc:648] HybridClock initialized: now 1786512324135054 us; error 35 us; skew 500 ppm
Aug 12 05:25:24 dist-test-slave-zpfg krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512324, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:24.138598 28970 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:25:24.139149 28970 webserver.cc:460] Webserver started at http://127.24.66.67:33151/ using document root <none> and password file <none>
I20260812 05:25:24.139406 28970 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:24.139478 28970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:24.139649 28970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:24.140793 28970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "30a62ce5c04c44b687f6f37e6922400f"
format_stamp: "Formatted at 2026-08-12 05:25:24 on dist-test-slave-zpfg"
server_key: "6020d8c7c65754c12860fec9c3dfad62"
server_key_iv: "93cd1f77397b26bbee5fe18e3223cc87"
server_key_version: "encryptionkey@0"
I20260812 05:25:24.141179 28970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "30a62ce5c04c44b687f6f37e6922400f"
format_stamp: "Formatted at 2026-08-12 05:25:24 on dist-test-slave-zpfg"
server_key: "6020d8c7c65754c12860fec9c3dfad62"
server_key_iv: "93cd1f77397b26bbee5fe18e3223cc87"
server_key_version: "encryptionkey@0"
I20260812 05:25:24.142741 28970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:24.143743 28986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:24.143986 28970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:24.144101 28970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "30a62ce5c04c44b687f6f37e6922400f"
format_stamp: "Formatted at 2026-08-12 05:25:24 on dist-test-slave-zpfg"
server_key: "6020d8c7c65754c12860fec9c3dfad62"
server_key_iv: "93cd1f77397b26bbee5fe18e3223cc87"
server_key_version: "encryptionkey@0"
I20260812 05:25:24.144192 28970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:24.162750 28970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:24.164848 28970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:24.165022 28970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:24.165297 28970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:24.165652 28970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:24.165707 28970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:24.165750 28970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:24.165804 28970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:24.172586 28970 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:33795
I20260812 05:25:24.172634 29108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:33795 every 8 connection(s)
I20260812 05:25:24.172983 28970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:24.174544 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 28970
I20260812 05:25:24.174638 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:24.174839 24841 external_mini_cluster.cc:1428] Setting key 4a0af2edec7d7eeb024ad4e3e9f58748
Aug 12 05:25:24 dist-test-slave-zpfg krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512324, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:24.183760 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:24.173721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:33593 (local address 127.24.66.126:40211)
0812 05:25:24.173803 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:24.173805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:24.174074 (+ 269us) server_negotiation.cc:407] Connection header received
0812 05:25:24.175362 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:24.175363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:24.175375 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:24.175414 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:24.176208 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.176719 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.177754 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.178005 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.178090 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:24.180505 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:24.180507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:24.180507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:24.180515 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:24.182183 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:24.182693 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:24.182695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:24.182695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:24.182761 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:24.183040 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:24.183041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:24.183041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:24.183171 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:24.183244 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:24.183579 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 05:25:24.183663 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 05:25:24.183965 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:24.173696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40211 (local address 127.24.66.67:33593)
0812 05:25:24.173916 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:24.173969 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:24.175239 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:24.175506 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:24.175512 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:24.175880 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:24.176104 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:24.176109 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.176826 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.176827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:24.177658 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:24.177660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.178102 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.178104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:24.178251 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:24.178797 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:24.178806 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:24.180396 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:24.182278 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:24.182287 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:24.182289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:24.182594 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:24.182846 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:24.182847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:24.182847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:24.182962 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:24.183246 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:24.183254 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:24.183436 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 05:25:24.183545 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 05:25:24.183715 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":113,"threads_started":1}
I20260812 05:25:24.184495 29109 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40211
I20260812 05:25:24.184597 29109 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:24.184747 29109 heartbeater.cc:511] Master 127.24.66.126:40211 requested a full tablet report, sending...
I20260812 05:25:24.185091 28614 ts_manager.cc:194] Registered new tserver with Master: 30a62ce5c04c44b687f6f37e6922400f (127.24.66.67:33795)
I20260812 05:25:24.185436 28614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:33593
I20260812 05:25:24.186870 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 05:25:24 dist-test-slave-zpfg krb5kdc[28566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 05:25:24 dist-test-slave-zpfg krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:25:24.212155 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:24.203849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57468 (local address 127.24.66.65:33731)
0812 05:25:24.204036 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:24.204042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:24.204056 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:25:24.204094 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:24.204096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:24.204111 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:24.204193 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:24.204818 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.205576 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.206358 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.206710 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.206797 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:24.209137 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:24.209146 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:24.209146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:24.209153 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:24.210809 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:24.211178 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:24.211188 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:24.211188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:24.211244 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:24.211520 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:24.211520 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:24.211521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:24.211671 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:24.211852 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:24.211933 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 05:25:24.212018 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
Aug 12 05:25:24 dist-test-slave-zpfg krb5kdc[28566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:24.221071 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:24.213653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39292 (local address 127.24.66.126:40211)
0812 05:25:24.213723 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:24.213725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:24.213733 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:24.213756 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:24.213757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:24.213765 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:24.213814 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:24.214406 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.214894 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.215593 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:24.215780 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:24.215839 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:24.218239 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:24.218241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:24.218241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:24.218248 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:24.219999 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:24.220394 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:24.220395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:24.220395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:24.220464 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:24.220622 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:24.220622 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:24.220622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:24.220763 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:24.220838 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:24.220936 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 05:25:24.221001 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 05:25:24.221350 28614 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39292
I20260812 05:25:24.221771 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 28680
I20260812 05:25:24.228643 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 28821
I20260812 05:25:24.234863 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 28970
I20260812 05:25:24.241904 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 28579
2026-08-12T05:25:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1995 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:24 dist-test-slave-zpfg krb5kdc[29122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:24 dist-test-slave-zpfg krb5kdc[29122](info): set up 2 sockets
Aug 12 05:25:24 dist-test-slave-zpfg krb5kdc[29122](info): commencing operation
krb5kdc: starting...
W20260812 05:25:25.329191 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:25 dist-test-slave-zpfg krb5kdc[29122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:25:25.466331 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35785
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:35785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:25.552380 29136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:25.552655 29136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:25.552698 29136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:25.553928 29136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:25.554003 29136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:25.554028 29136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:25.554049 29136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:25.554080 29136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:25.555857 29136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:35785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:25.556191 29136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:25.556490 29136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:25.561822 29147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:25.562001 29143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:25.562022 29144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:25.561941 29136 server_base.cc:1034] running on GCE node
I20260812 05:25:25.562381 29136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:25.562719 29136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:25.563895 29136 hybrid_clock.cc:648] HybridClock initialized: now 1786512325563855 us; error 74 us; skew 500 ppm
Aug 12 05:25:25 dist-test-slave-zpfg krb5kdc[29122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512325, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:25.566641 29136 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:25.567024 29136 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:25.567102 29136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:25.567262 29136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:25.568379 29136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "6ebc8f1d3def452e99b87d643b929c82"
format_stamp: "Formatted at 2026-08-12 05:25:25 on dist-test-slave-zpfg"
server_key: "d42e83468e499acb6653eeecd2a35207"
server_key_iv: "16ba70cc49ec6903d5e13ec144409944"
server_key_version: "encryptionkey@0"
I20260812 05:25:25.568771 29136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "6ebc8f1d3def452e99b87d643b929c82"
format_stamp: "Formatted at 2026-08-12 05:25:25 on dist-test-slave-zpfg"
server_key: "d42e83468e499acb6653eeecd2a35207"
server_key_iv: "16ba70cc49ec6903d5e13ec144409944"
server_key_version: "encryptionkey@0"
I20260812 05:25:25.570391 29136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:25:25.571291 29153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:25.571552 29136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:25.571687 29136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "6ebc8f1d3def452e99b87d643b929c82"
format_stamp: "Formatted at 2026-08-12 05:25:25 on dist-test-slave-zpfg"
server_key: "d42e83468e499acb6653eeecd2a35207"
server_key_iv: "16ba70cc49ec6903d5e13ec144409944"
server_key_version: "encryptionkey@0"
I20260812 05:25:25.571789 29136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:25.583681 29136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:25.585887 29136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:25.586072 29136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:25.590287 29136 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:35785
I20260812 05:25:25.590430 29205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:35785 every 8 connection(s)
I20260812 05:25:25.590745 29136 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:25.591845 29206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:25.592448 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 29136
I20260812 05:25:25.592574 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:25.592763 24841 external_mini_cluster.cc:1428] Setting key fe04a96ca463b0e14c79c4c6f889782d
I20260812 05:25:25.595110 29206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82: Bootstrap starting.
I20260812 05:25:25.595783 29206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:25.596078 29206 log.cc:824] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:25.596925 29206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82: No bootstrap required, opened a new log
I20260812 05:25:25.599614 29206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebc8f1d3def452e99b87d643b929c82" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35785 } }
I20260812 05:25:25.599767 29206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:25.599790 29206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebc8f1d3def452e99b87d643b929c82, State: Initialized, Role: FOLLOWER
I20260812 05:25:25.599934 29206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebc8f1d3def452e99b87d643b929c82" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35785 } }
I20260812 05:25:25.600010 29206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:25.600066 29206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:25.600116 29206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 05:25:25 dist-test-slave-zpfg krb5kdc[29122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:25.600859 29206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebc8f1d3def452e99b87d643b929c82" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35785 } }
I20260812 05:25:25.600980 29206 leader_election.cc:302] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ebc8f1d3def452e99b87d643b929c82; no voters:
I20260812 05:25:25.601172 29206 leader_election.cc:288] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:25.601250 29211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:25.601466 29211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [term 1 LEADER]: Becoming Leader. State: Replica: 6ebc8f1d3def452e99b87d643b929c82, State: Running, Role: LEADER
I20260812 05:25:25.601464 29206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:25.601672 29211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebc8f1d3def452e99b87d643b929c82" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35785 } }
I20260812 05:25:25.602828 29213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ebc8f1d3def452e99b87d643b929c82. Latest consensus state: current_term: 1 leader_uuid: "6ebc8f1d3def452e99b87d643b929c82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebc8f1d3def452e99b87d643b929c82" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35785 } } }
I20260812 05:25:25.602919 29213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:25.602075 29212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ebc8f1d3def452e99b87d643b929c82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebc8f1d3def452e99b87d643b929c82" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35785 } } }
I20260812 05:25:25.603366 29212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82 [sys.catalog]: This master's current role is: LEADER
W20260812 05:25:25.603497 29227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:25:25.603526 29227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:25:25.603595 29228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:25.603777 29228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:25.605047 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:25.593344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60690 (local address 127.24.66.126:35785)
0812 05:25:25.593528 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:25.593536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:25.593563 (+ 27us) server_negotiation.cc:407] Connection header received
0812 05:25:25.593980 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:25.593987 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:25.594044 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:25.594165 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:25.595022 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:25.596068 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:25.597221 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:25.597645 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:25.597733 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:25.600628 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:25.600633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:25.600633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:25.600645 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:25.603211 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:25.603670 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:25.603674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:25.603674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:25.603795 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:25.604054 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:25.604056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:25.604056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:25.604398 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:25.604541 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:25.604668 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 05:25:25.604830 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260812 05:25:25.605472 29228 catalog_manager.cc:1347] Generated new cluster ID: 6d9d3dcccaa44fb49ccfb64fcd7ff7c6
I20260812 05:25:25.605525 29228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:25.616580 29228 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:25.617177 29228 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:25.628360 29228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82: Generated new TSK 0
I20260812 05:25:25.628618 29228 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:25:25.662046 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35785
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:25.757547 29233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:25.757800 29233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:25.757830 29233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:25.759302 29233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:25.759363 29233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:25.759397 29233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:25.761193 29233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:25.761483 29233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:25.761777 29233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:25.767755 29238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:25.767802 29233 server_base.cc:1034] running on GCE node
W20260812 05:25:25.767773 29241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:25.767781 29239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:25.768190 29233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:25.768460 29233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:25.769680 29233 hybrid_clock.cc:648] HybridClock initialized: now 1786512325769656 us; error 42 us; skew 500 ppm
Aug 12 05:25:25 dist-test-slave-zpfg krb5kdc[29122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512325, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:25.772658 29233 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:25:25.773047 29233 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:25.773102 29233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:25.773276 29233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:25.774437 29233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "50826ec00350425bb785515f0b2cfc7c"
format_stamp: "Formatted at 2026-08-12 05:25:25 on dist-test-slave-zpfg"
server_key: "5f79aad5c771b4f21ebee42b30ef5b51"
server_key_iv: "3e486de3b9b47fe036d3a71290ca1fec"
server_key_version: "encryptionkey@0"
I20260812 05:25:25.774781 29233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "50826ec00350425bb785515f0b2cfc7c"
format_stamp: "Formatted at 2026-08-12 05:25:25 on dist-test-slave-zpfg"
server_key: "5f79aad5c771b4f21ebee42b30ef5b51"
server_key_iv: "3e486de3b9b47fe036d3a71290ca1fec"
server_key_version: "encryptionkey@0"
I20260812 05:25:25.776343 29233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:25:25.777351 29247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:25.777558 29233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:25.777648 29233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "50826ec00350425bb785515f0b2cfc7c"
format_stamp: "Formatted at 2026-08-12 05:25:25 on dist-test-slave-zpfg"
server_key: "5f79aad5c771b4f21ebee42b30ef5b51"
server_key_iv: "3e486de3b9b47fe036d3a71290ca1fec"
server_key_version: "encryptionkey@0"
I20260812 05:25:25.777716 29233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:25.796487 29233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:25.798988 29233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:25.799117 29233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:25.799331 29233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:25.799685 29233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:25.799721 29233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:25.799746 29233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:25.799793 29233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:25.806150 29233 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:45647
I20260812 05:25:25.806241 29370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:45647 every 8 connection(s)
I20260812 05:25:25.806571 29233 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:25.809209 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 29233
I20260812 05:25:25.809336 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:25.809542 24841 external_mini_cluster.cc:1428] Setting key 755380ffed5b9ed83494ce011ac5717b
Aug 12 05:25:25 dist-test-slave-zpfg krb5kdc[29122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512325, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:25.817245 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:25.807436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:33401 (local address 127.24.66.126:35785)
0812 05:25:25.807485 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:25.807487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:25.807635 (+ 148us) server_negotiation.cc:407] Connection header received
0812 05:25:25.808139 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:25.808140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:25.808150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:25.808213 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:25.809525 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:25.810172 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:25.811338 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:25.811545 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:25.811631 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:25.813918 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:25.813920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:25.813920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:25.813930 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:25.815794 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:25.816230 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:25.816231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:25.816232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:25.816300 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:25.816545 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:25.816546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:25.816547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:25.816695 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:25.816769 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:25.817031 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 05:25:25.817134 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:25:25.817351 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:25.807324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35785 (local address 127.24.66.65:33401)
0812 05:25:25.807516 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:25.807561 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:25.808048 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:25.808293 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:25.808302 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:25.808581 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:25.809402 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:25.809411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:25.810280 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:25.810281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:25.811181 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:25.811184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:25.811645 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:25.811646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:25.811796 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:25.812382 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:25.812391 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:25.813817 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:25.815902 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:25.815910 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:25.815912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:25.816152 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:25.816372 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:25.816373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:25.816373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:25.816479 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:25.816763 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:25.816769 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:25.816896 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 05:25:25.816981 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 05:25:25.817107 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260812 05:25:25.817931 29371 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35785
I20260812 05:25:25.818053 29371 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:25.818231 29371 heartbeater.cc:511] Master 127.24.66.126:35785 requested a full tablet report, sending...
I20260812 05:25:25.818969 29170 ts_manager.cc:194] Registered new tserver with Master: 50826ec00350425bb785515f0b2cfc7c (127.24.66.65:45647)
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
I20260812 05:25:25.819888 29170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:33401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:25:25.841727 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35785
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:25.928105 29376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:25.928351 29376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:25.928380 29376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:25.929611 29376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:25.929658 29376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:25.929687 29376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:25.931361 29376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:25.931589 29376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:25.931803 29376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:25.937537 29382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:25.937676 29384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:25.937704 29381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:25.937986 29376 server_base.cc:1034] running on GCE node
I20260812 05:25:25.938284 29376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:25.938608 29376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:25.939851 29376 hybrid_clock.cc:648] HybridClock initialized: now 1786512325939842 us; error 55 us; skew 500 ppm
Aug 12 05:25:25 dist-test-slave-zpfg krb5kdc[29122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512325, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:25.942854 29376 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:25:25.943222 29376 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:25.943276 29376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:25.943377 29376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:25.944458 29376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "80abb22c0df44756814b91e97ac8b865"
format_stamp: "Formatted at 2026-08-12 05:25:25 on dist-test-slave-zpfg"
server_key: "248e0ac569bcc6462b4013cc2c982eaa"
server_key_iv: "6a7c62301146496a1bcf21b0b8decd9b"
server_key_version: "encryptionkey@0"
I20260812 05:25:25.944784 29376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "80abb22c0df44756814b91e97ac8b865"
format_stamp: "Formatted at 2026-08-12 05:25:25 on dist-test-slave-zpfg"
server_key: "248e0ac569bcc6462b4013cc2c982eaa"
server_key_iv: "6a7c62301146496a1bcf21b0b8decd9b"
server_key_version: "encryptionkey@0"
I20260812 05:25:25.946171 29376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:25.947117 29390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:25.947409 29376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:25.947511 29376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "80abb22c0df44756814b91e97ac8b865"
format_stamp: "Formatted at 2026-08-12 05:25:25 on dist-test-slave-zpfg"
server_key: "248e0ac569bcc6462b4013cc2c982eaa"
server_key_iv: "6a7c62301146496a1bcf21b0b8decd9b"
server_key_version: "encryptionkey@0"
I20260812 05:25:25.947573 29376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:25.965915 29376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:25.968484 29376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:25.968619 29376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:25.968899 29376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:25.969301 29376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:25.969343 29376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:25.969368 29376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:25.969383 29376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:25.976029 29376 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:38515
I20260812 05:25:25.976066 29509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:38515 every 8 connection(s)
I20260812 05:25:25.976414 29376 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:25.978068 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 29376
I20260812 05:25:25.978197 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:25.978442 24841 external_mini_cluster.cc:1428] Setting key 0ea420ef4396ec6c016a39e606b20480
Aug 12 05:25:25 dist-test-slave-zpfg krb5kdc[29122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512325, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
I20260812 05:25:25.987733 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:25.977165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:33097 (local address 127.24.66.126:35785)
0812 05:25:25.977244 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:25.977246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:25.977486 (+ 240us) server_negotiation.cc:407] Connection header received
0812 05:25:25.978024 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:25.978025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:25.978035 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:25.978095 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:25.979356 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:25.979915 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:25.981397 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:25.981828 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:25.981933 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:25.984763 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:25.984764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:25.984765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:25.984773 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:25.986257 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:25.986708 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:25.986710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:25.986710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:25.986774 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:25.987052 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:25.987053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:25.987053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:25.987173 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:25.987247 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:25.987546 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 05:25:25.987638 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:25:25.987974 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:25.977162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35785 (local address 127.24.66.66:33097)
0812 05:25:25.977347 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:25.977391 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:25.977916 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:25.978199 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:25.978204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:25.978661 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:25.979230 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:25.979238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:25.980015 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:25.980016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:25.981298 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:25.981301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:25.981969 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:25.981970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:25.982106 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:25.982819 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:25.982828 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:25.984649 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:25.986374 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:25.986383 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:25.986386 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:25.986626 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:25.986864 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:25.986865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:25.986865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:25.986978 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:25.987249 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:25.987257 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:25.987409 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 05:25:25.987512 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 05:25:25.987672 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260812 05:25:25.988550 29510 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35785
I20260812 05:25:25.988652 29510 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:25.988781 29510 heartbeater.cc:511] Master 127.24.66.126:35785 requested a full tablet report, sending...
I20260812 05:25:25.989269 29170 ts_manager.cc:194] Registered new tserver with Master: 80abb22c0df44756814b91e97ac8b865 (127.24.66.66:38515)
I20260812 05:25:25.989651 29170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:33097
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:25:26.010488 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35785
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:26.103988 29515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:26.104207 29515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:26.104241 29515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:26.105437 29515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:26.105482 29515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:26.105509 29515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:26.107230 29515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:26.107483 29515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:26.107738 29515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:26.113457 29524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:26.113444 29521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:26.113555 29522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:26.113659 29515 server_base.cc:1034] running on GCE node
I20260812 05:25:26.113970 29515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:26.114254 29515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:26.115417 29515 hybrid_clock.cc:648] HybridClock initialized: now 1786512326115377 us; error 51 us; skew 500 ppm
Aug 12 05:25:26 dist-test-slave-zpfg krb5kdc[29122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512326, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:26.118431 29515 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:25:26.118839 29515 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:26.118904 29515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:26.119004 29515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:26.120055 29515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "fc6a43724aeb4b3b9ef695492a34c8b0"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "fb9fd45926485b53ee0cb4b92b3e4aad"
server_key_iv: "3c05b354a8461c4fa8c4496abf5b5bee"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.120370 29515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "fc6a43724aeb4b3b9ef695492a34c8b0"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "fb9fd45926485b53ee0cb4b92b3e4aad"
server_key_iv: "3c05b354a8461c4fa8c4496abf5b5bee"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.121748 29515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:25:26.122606 29531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:26.122825 29515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:26.122905 29515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "fc6a43724aeb4b3b9ef695492a34c8b0"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "fb9fd45926485b53ee0cb4b92b3e4aad"
server_key_iv: "3c05b354a8461c4fa8c4496abf5b5bee"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.122963 29515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:26.135082 29515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:26.137521 29515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:26.137704 29515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:26.137928 29515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:26.138285 29515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:26.138329 29515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:26.138411 29515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:26.138450 29515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:26.144928 29515 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:41177
I20260812 05:25:26.144964 29650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:41177 every 8 connection(s)
I20260812 05:25:26.145262 29515 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:26.147261 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 29515
I20260812 05:25:26.147367 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:26.147563 24841 external_mini_cluster.cc:1428] Setting key d1b5fe730c627179c4269e9301146087
Aug 12 05:25:26 dist-test-slave-zpfg krb5kdc[29122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512326, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:26.156018 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.146030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:34461 (local address 127.24.66.126:35785)
0812 05:25:26.146079 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:26.146081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:26.146386 (+ 305us) server_negotiation.cc:407] Connection header received
0812 05:25:26.146911 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:26.146912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:26.146922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:26.146978 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:26.148331 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.148918 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.150010 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.150191 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.150244 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.152737 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:26.152738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:26.152738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:26.152746 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:26.154616 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:26.155044 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:26.155045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:26.155046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:26.155106 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:26.155388 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:26.155389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:26.155389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:26.155502 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:26.155557 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:26.155862 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 05:25:26.155949 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:25:26.156126 29653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.146007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35785 (local address 127.24.66.67:34461)
0812 05:25:26.146168 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:26.146208 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:26.146817 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:26.147038 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:26.147041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:26.147478 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:26.148241 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.148249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.149006 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.149007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.149928 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.149931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.150271 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.150272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.150461 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.151032 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:26.151041 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:26.152621 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:26.154700 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:26.154708 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:26.154710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:26.154955 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:26.155172 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:26.155173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:26.155174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:26.155289 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:26.155561 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:26.155566 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:26.155690 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 05:25:26.155804 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 05:25:26.155934 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260812 05:25:26.156684 29651 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35785
I20260812 05:25:26.156802 29651 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:26.156971 29651 heartbeater.cc:511] Master 127.24.66.126:35785 requested a full tablet report, sending...
I20260812 05:25:26.157449 29170 ts_manager.cc:194] Registered new tserver with Master: fc6a43724aeb4b3b9ef695492a34c8b0 (127.24.66.67:41177)
I20260812 05:25:26.157861 29170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:34461
I20260812 05:25:26.159488 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:25:26.167313 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.160725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60700 (local address 127.24.66.126:35785)
0812 05:25:26.160804 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:26.160806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:26.160945 (+ 139us) server_negotiation.cc:407] Connection header received
0812 05:25:26.160974 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:26.160974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:26.160985 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:26.161043 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:26.161628 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.162110 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.162913 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.163094 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.163150 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.164442 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:26.164443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:26.164443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:26.164450 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:26.166161 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:26.166543 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:26.166543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:26.166544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:26.166593 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:26.166859 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:26.166859 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:26.166859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:26.166997 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:26.167070 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:26.167185 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 05:25:26.167246 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 05:25:26.169143 29170 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:26.171435 29170 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:26.178833 29663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.173980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:41177 (local address 127.0.0.1:35480)
0812 05:25:26.175167 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:26.175179 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:26.175229 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:26.175531 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:26.175532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:26.175535 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:26.175590 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.175592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.176897 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.176899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.177705 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.177707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.178487 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.178488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.178574 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.178591 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 05:25:26.178678 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 05:25:26.178737 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"thread_start_us":114,"threads_started":1}
I20260812 05:25:26.178833 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.174266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35480 (local address 127.24.66.67:41177)
0812 05:25:26.174520 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:26.174525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:26.175299 (+ 774us) server_negotiation.cc:407] Connection header received
0812 05:25:26.175346 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:26.175348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:26.175365 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:26.175436 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:26.175670 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.176813 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.177817 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.178397 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.178470 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.178532 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:26.178701 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 05:25:26.178740 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":145,"threads_started":1}
I20260812 05:25:26.179756 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.173881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:45647 (local address 127.0.0.1:36620)
0812 05:25:26.174167 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:26.174211 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:26.174283 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:26.174740 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:26.174744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:26.174747 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:26.174826 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.174828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.176684 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.176684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.178250 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.178253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.179507 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.179508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.179574 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.179578 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:26.179636 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 05:25:26.179681 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":82,"spinlock_wait_cycles":896,"thread_start_us":95,"threads_started":1}
I20260812 05:25:26.179777 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.174209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36620 (local address 127.24.66.65:45647)
0812 05:25:26.174520 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:26.174525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:26.174536 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:26.174584 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:26.174586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:26.174607 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:26.174692 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:26.174930 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.176562 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.178376 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.179401 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.179502 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.179560 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:26.179650 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 05:25:26.179687 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":181,"threads_started":1}
I20260812 05:25:26.179728 29580 tablet_service.cc:1467] Processing CreateTablet for tablet 14fa5630e57f4ada936b8eefbd8f5b9e (DEFAULT_TABLE table=test-table [id=69fc728399714e1e845ea4b9665e821d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:26.180202 29580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14fa5630e57f4ada936b8eefbd8f5b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:26.180234 29666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.174049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50332 (local address 127.24.66.66:38515)
0812 05:25:26.174876 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:26.174881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:26.174892 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:26.174935 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:26.174937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:26.174952 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:26.175023 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:26.175212 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.176531 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.178389 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.179462 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.179540 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.179600 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:26.180035 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 05:25:26.180131 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":73,"threads_started":1}
I20260812 05:25:26.180473 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.173881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:38515 (local address 127.0.0.1:50332)
0812 05:25:26.174230 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:26.174242 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:26.174304 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:26.175051 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:26.175052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:26.175054 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:26.175107 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.175109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.176673 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.176674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.178250 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.178253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.179654 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.179655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.179741 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.179745 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:26.179803 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 05:25:26.180398 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":55,"thread_start_us":80,"threads_started":1}
I20260812 05:25:26.180727 29300 tablet_service.cc:1467] Processing CreateTablet for tablet 14fa5630e57f4ada936b8eefbd8f5b9e (DEFAULT_TABLE table=test-table [id=69fc728399714e1e845ea4b9665e821d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:26.181093 29300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14fa5630e57f4ada936b8eefbd8f5b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:26.181269 29440 tablet_service.cc:1467] Processing CreateTablet for tablet 14fa5630e57f4ada936b8eefbd8f5b9e (DEFAULT_TABLE table=test-table [id=69fc728399714e1e845ea4b9665e821d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:26.181618 29440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14fa5630e57f4ada936b8eefbd8f5b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:26.183452 29669 tablet_bootstrap.cc:492] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: Bootstrap starting.
I20260812 05:25:26.184072 29669 tablet_bootstrap.cc:654] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:26.184742 29670 tablet_bootstrap.cc:492] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c: Bootstrap starting.
I20260812 05:25:26.184769 29669 log.cc:824] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:26.185499 29670 tablet_bootstrap.cc:654] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:26.185600 29669 tablet_bootstrap.cc:492] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: No bootstrap required, opened a new log
I20260812 05:25:26.185639 29671 tablet_bootstrap.cc:492] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: Bootstrap starting.
I20260812 05:25:26.185717 29669 ts_tablet_manager.cc:1397] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:25:26.186187 29671 tablet_bootstrap.cc:654] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:26.186194 29670 log.cc:824] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:26.186822 29671 log.cc:824] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:26.187067 29670 tablet_bootstrap.cc:492] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c: No bootstrap required, opened a new log
I20260812 05:25:26.187124 29670 ts_tablet_manager.cc:1397] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:25:26.187672 29671 tablet_bootstrap.cc:492] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: No bootstrap required, opened a new log
I20260812 05:25:26.187726 29671 ts_tablet_manager.cc:1397] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:25:26.187929 29669 raft_consensus.cc:348] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80abb22c0df44756814b91e97ac8b865" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38515 } } peers { permanent_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41177 } } peers { permanent_uuid: "50826ec00350425bb785515f0b2cfc7c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45647 } }
I20260812 05:25:26.188066 29669 raft_consensus.cc:374] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:26.188089 29669 raft_consensus.cc:729] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6a43724aeb4b3b9ef695492a34c8b0, State: Initialized, Role: FOLLOWER
I20260812 05:25:26.188225 29669 consensus_queue.cc:260] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80abb22c0df44756814b91e97ac8b865" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38515 } } peers { permanent_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41177 } } peers { permanent_uuid: "50826ec00350425bb785515f0b2cfc7c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45647 } }
I20260812 05:25:26.188535 29651 heartbeater.cc:503] Master 127.24.66.126:35785 was elected leader, sending a full tablet report...
I20260812 05:25:26.188537 29669 ts_tablet_manager.cc:1428] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:25:26.189168 29670 raft_consensus.cc:348] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80abb22c0df44756814b91e97ac8b865" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38515 } } peers { permanent_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41177 } } peers { permanent_uuid: "50826ec00350425bb785515f0b2cfc7c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45647 } }
I20260812 05:25:26.189337 29670 raft_consensus.cc:374] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:26.189374 29670 raft_consensus.cc:729] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50826ec00350425bb785515f0b2cfc7c, State: Initialized, Role: FOLLOWER
I20260812 05:25:26.189489 29670 consensus_queue.cc:260] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80abb22c0df44756814b91e97ac8b865" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38515 } } peers { permanent_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41177 } } peers { permanent_uuid: "50826ec00350425bb785515f0b2cfc7c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45647 } }
I20260812 05:25:26.189781 29670 ts_tablet_manager.cc:1428] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:25:26.189788 29371 heartbeater.cc:503] Master 127.24.66.126:35785 was elected leader, sending a full tablet report...
I20260812 05:25:26.189918 29671 raft_consensus.cc:348] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80abb22c0df44756814b91e97ac8b865" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38515 } } peers { permanent_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41177 } } peers { permanent_uuid: "50826ec00350425bb785515f0b2cfc7c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45647 } }
I20260812 05:25:26.190085 29671 raft_consensus.cc:374] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:26.190125 29671 raft_consensus.cc:729] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80abb22c0df44756814b91e97ac8b865, State: Initialized, Role: FOLLOWER
I20260812 05:25:26.190270 29671 consensus_queue.cc:260] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80abb22c0df44756814b91e97ac8b865" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38515 } } peers { permanent_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41177 } } peers { permanent_uuid: "50826ec00350425bb785515f0b2cfc7c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45647 } }
I20260812 05:25:26.190577 29671 ts_tablet_manager.cc:1428] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 05:25:26.190603 29510 heartbeater.cc:503] Master 127.24.66.126:35785 was elected leader, sending a full tablet report...
W20260812 05:25:26.227006 29511 tablet.cc:2369] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:25:26.307361 29372 tablet.cc:2369] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:25:26.395864 29652 tablet.cc:2369] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:25:26.453272 29677 raft_consensus.cc:482] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:26.453473 29677 raft_consensus.cc:504] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80abb22c0df44756814b91e97ac8b865" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38515 } } peers { permanent_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41177 } } peers { permanent_uuid: "50826ec00350425bb785515f0b2cfc7c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45647 } }
I20260812 05:25:26.453917 29677 leader_election.cc:288] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc6a43724aeb4b3b9ef695492a34c8b0 (127.24.66.67:41177), 50826ec00350425bb785515f0b2cfc7c (127.24.66.65:45647)
I20260812 05:25:26.457166 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.454276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:54841 (local address 127.24.66.67:41177)
0812 05:25:26.454369 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:26.454371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:26.454387 (+ 16us) server_negotiation.cc:407] Connection header received
0812 05:25:26.454429 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:26.454429 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:26.454449 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:26.454531 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:26.454780 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.455368 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.456364 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.456786 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.456845 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.456852 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:26.457047 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 05:25:26.457092 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:25:26.457172 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.454228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:51117 (local address 127.24.66.65:45647)
0812 05:25:26.454323 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:26.454326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:26.454527 (+ 201us) server_negotiation.cc:407] Connection header received
0812 05:25:26.454605 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:26.454606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:26.454617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:26.454686 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:26.454942 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.455443 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.456442 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.456789 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.456899 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.456907 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:26.457075 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 05:25:26.457110 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 05:25:26.457170 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.454153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:41177 (local address 127.24.66.66:54841)
0812 05:25:26.454210 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:26.454225 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:26.454281 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:26.454587 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:26.454588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:26.454591 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:26.454675 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.454677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.455460 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.455462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.456254 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.456256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.456875 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.456877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.456969 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.456972 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:26.457015 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 05:25:26.457083 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 05:25:26.457196 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.454164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:45647 (local address 127.24.66.66:51117)
0812 05:25:26.454436 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:26.454448 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:26.454520 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:26.454779 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:26.454780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:26.454783 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:26.454855 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.454856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.455528 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.455529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.456356 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:26.456359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.456915 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.456918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:26.456977 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.456979 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:25:26.457061 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 05:25:26.457099 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":97,"thread_start_us":92,"threads_started":1}
I20260812 05:25:26.457614 29323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fa5630e57f4ada936b8eefbd8f5b9e" candidate_uuid: "80abb22c0df44756814b91e97ac8b865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50826ec00350425bb785515f0b2cfc7c" is_pre_election: true
I20260812 05:25:26.457597 29602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fa5630e57f4ada936b8eefbd8f5b9e" candidate_uuid: "80abb22c0df44756814b91e97ac8b865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" is_pre_election: true
I20260812 05:25:26.457780 29602 raft_consensus.cc:2454] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80abb22c0df44756814b91e97ac8b865 in term 0.
I20260812 05:25:26.457780 29323 raft_consensus.cc:2454] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80abb22c0df44756814b91e97ac8b865 in term 0.
I20260812 05:25:26.458058 29391 leader_election.cc:302] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 50826ec00350425bb785515f0b2cfc7c, 80abb22c0df44756814b91e97ac8b865, fc6a43724aeb4b3b9ef695492a34c8b0; no voters:
I20260812 05:25:26.458205 29677 raft_consensus.cc:2781] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:26.458276 29677 raft_consensus.cc:482] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:26.458300 29677 raft_consensus.cc:3037] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:26.459082 29677 raft_consensus.cc:504] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80abb22c0df44756814b91e97ac8b865" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38515 } } peers { permanent_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41177 } } peers { permanent_uuid: "50826ec00350425bb785515f0b2cfc7c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45647 } }
I20260812 05:25:26.459331 29677 leader_election.cc:288] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [CANDIDATE]: Term 1 election: Requested vote from peers fc6a43724aeb4b3b9ef695492a34c8b0 (127.24.66.67:41177), 50826ec00350425bb785515f0b2cfc7c (127.24.66.65:45647)
I20260812 05:25:26.459414 29602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fa5630e57f4ada936b8eefbd8f5b9e" candidate_uuid: "80abb22c0df44756814b91e97ac8b865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0"
I20260812 05:25:26.459442 29323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14fa5630e57f4ada936b8eefbd8f5b9e" candidate_uuid: "80abb22c0df44756814b91e97ac8b865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50826ec00350425bb785515f0b2cfc7c"
I20260812 05:25:26.459506 29602 raft_consensus.cc:3037] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:26.459561 29323 raft_consensus.cc:3037] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:26.460392 29602 raft_consensus.cc:2454] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80abb22c0df44756814b91e97ac8b865 in term 1.
I20260812 05:25:26.460424 29323 raft_consensus.cc:2454] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80abb22c0df44756814b91e97ac8b865 in term 1.
I20260812 05:25:26.460592 29391 leader_election.cc:302] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50826ec00350425bb785515f0b2cfc7c, 80abb22c0df44756814b91e97ac8b865; no voters:
I20260812 05:25:26.460750 29677 raft_consensus.cc:2781] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:26.460937 29677 raft_consensus.cc:686] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 1 LEADER]: Becoming Leader. State: Replica: 80abb22c0df44756814b91e97ac8b865, State: Running, Role: LEADER
I20260812 05:25:26.461086 29677 consensus_queue.cc:237] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80abb22c0df44756814b91e97ac8b865" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38515 } } peers { permanent_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41177 } } peers { permanent_uuid: "50826ec00350425bb785515f0b2cfc7c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45647 } }
I20260812 05:25:26.461967 29169 catalog_manager.cc:5495] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80abb22c0df44756814b91e97ac8b865 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "80abb22c0df44756814b91e97ac8b865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80abb22c0df44756814b91e97ac8b865" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50826ec00350425bb785515f0b2cfc7c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45647 } health_report { overall_health: UNKNOWN } } }
I20260812 05:25:26.467780 29666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.465456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50336 (local address 127.24.66.66:38515)
0812 05:25:26.465495 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:26.465497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:26.465508 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:26.465581 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:26.465582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:26.465597 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:26.465655 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:26.465800 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.466264 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.467077 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:26.467258 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:26.467313 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:26.467457 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:26.467550 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:26.467610 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:26.467640 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 05:25:26.467657 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:25:26.469779 29323 raft_consensus.cc:1261] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c [term 1 FOLLOWER]: Refusing update from remote peer 80abb22c0df44756814b91e97ac8b865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:26.470170 29602 raft_consensus.cc:1261] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0 [term 1 FOLLOWER]: Refusing update from remote peer 80abb22c0df44756814b91e97ac8b865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:26.470207 29679 consensus_queue.cc:1035] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50826ec00350425bb785515f0b2cfc7c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:26.470666 29679 consensus_queue.cc:1035] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc6a43724aeb4b3b9ef695492a34c8b0" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:26.472759 29682 mvcc.cc:205] Tried to move back new op lower bound from 7317554489216733184 to 7317554489185181696. Current Snapshot: MvccSnapshot[applied={T|T < 7317554489216733184}]
I20260812 05:25:26.475754 29169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:26.475868 29169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:26.476840 29169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ebc8f1d3def452e99b87d643b929c82: Sending DeleteTablet for 3 replicas of tablet 14fa5630e57f4ada936b8eefbd8f5b9e
I20260812 05:25:26.477159 29440 tablet_service.cc:1514] Processing DeleteTablet for tablet 14fa5630e57f4ada936b8eefbd8f5b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:26 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:50332
I20260812 05:25:26.477210 29300 tablet_service.cc:1514] Processing DeleteTablet for tablet 14fa5630e57f4ada936b8eefbd8f5b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:26 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:36620
I20260812 05:25:26.477488 29693 tablet_replica.cc:330] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c: stopping tablet replica
I20260812 05:25:26.477653 29693 raft_consensus.cc:2229] T 14fa5630e57f4ada936b8eefbd8f5b9e P 50826ec00350425bb785515f0b2cfc7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:26.477648 29580 tablet_service.cc:1514] Processing DeleteTablet for tablet 14fa5630e57f4ada936b8eefbd8f5b9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:26 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:35480
I20260812 05:25:26.477795 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 29233
I20260812 05:25:26.480445 29694 tablet_replica.cc:330] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: stopping tablet replica
I20260812 05:25:26.480598 29694 raft_consensus.cc:2229] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:26.480724 29694 raft_consensus.cc:2258] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:26.481166 29694 ts_tablet_manager.cc:1905] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:26.482877 29694 ts_tablet_manager.cc:1918] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:26.482972 29694 log.cc:1196] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/14fa5630e57f4ada936b8eefbd8f5b9e
I20260812 05:25:26.483251 29694 ts_tablet_manager.cc:1939] T 14fa5630e57f4ada936b8eefbd8f5b9e P fc6a43724aeb4b3b9ef695492a34c8b0: Deleting consensus metadata
I20260812 05:25:26.483750 29157 catalog_manager.cc:4841] TS fc6a43724aeb4b3b9ef695492a34c8b0 (127.24.66.67:41177): tablet 14fa5630e57f4ada936b8eefbd8f5b9e (table test-table [id=69fc728399714e1e845ea4b9665e821d]) successfully deleted
I20260812 05:25:26.486930 29692 tablet_replica.cc:330] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: stopping tablet replica
I20260812 05:25:26.487121 29692 raft_consensus.cc:2229] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:26.487294 29692 raft_consensus.cc:2258] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:26.487843 29692 ts_tablet_manager.cc:1905] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:26.489660 29692 ts_tablet_manager.cc:1918] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:26.489758 29692 log.cc:1196] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/14fa5630e57f4ada936b8eefbd8f5b9e
W20260812 05:25:26.489979 29154 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:45647 (error 108)
I20260812 05:25:26.490110 29692 ts_tablet_manager.cc:1939] T 14fa5630e57f4ada936b8eefbd8f5b9e P 80abb22c0df44756814b91e97ac8b865: Deleting consensus metadata
I20260812 05:25:26.490633 29155 catalog_manager.cc:4841] TS 80abb22c0df44756814b91e97ac8b865 (127.24.66.66:38515): tablet 14fa5630e57f4ada936b8eefbd8f5b9e (table test-table [id=69fc728399714e1e845ea4b9665e821d]) successfully deleted
I20260812 05:25:26.490736 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.490532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:45647 (local address 127.0.0.1:36632)
0812 05:25:26.490585 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:26.490631 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:45647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 05:25:26.490846 29154 catalog_manager.cc:4584] TS 50826ec00350425bb785515f0b2cfc7c (127.24.66.65:45647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14fa5630e57f4ada936b8eefbd8f5b9e: Network error: Client connection negotiation failed: client connection to 127.24.66.65:45647: connect: Connection refused (error 111)
I20260812 05:25:26.490937 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 29376
I20260812 05:25:26.499396 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 29515
I20260812 05:25:26.506289 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 29136
2026-08-12T05:25:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2264 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T05:25:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:26Z Disabled control of system clock
I20260812 05:25:26.538601 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:32859
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:32859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 05:25:26.624706 29700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:26.624925 29700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:26.624948 29700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:26.626204 29700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:26.626258 29700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:26.626271 29700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:26.626286 29700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:26.626297 29700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:26.627786 29700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:32859
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:32859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:26.628098 29700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:26.628427 29700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:26.633708 29708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:26.633715 29705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:26.633819 29706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:26.634109 29700 server_base.cc:1034] running on GCE node
I20260812 05:25:26.634454 29700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:26.634780 29700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:26.635963 29700 hybrid_clock.cc:648] HybridClock initialized: now 1786512326635938 us; error 47 us; skew 500 ppm
I20260812 05:25:26.637648 29700 webserver.cc:460] Webserver started at http://127.24.66.126:44603/ using document root <none> and password file <none>
I20260812 05:25:26.637989 29700 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:26.638051 29700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:26.638211 29700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:26.639557 29700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "b114957c126549aca201a914a7f69cce"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "c504461722f627e844b2893c1a4e8fc6"
server_key_iv: "f8af38a1293907ab15053b7883aab248"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.640010 29700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "b114957c126549aca201a914a7f69cce"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "c504461722f627e844b2893c1a4e8fc6"
server_key_iv: "f8af38a1293907ab15053b7883aab248"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.641544 29700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 05:25:26.642549 29715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:26.642752 29700 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:26.642858 29700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "b114957c126549aca201a914a7f69cce"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "c504461722f627e844b2893c1a4e8fc6"
server_key_iv: "f8af38a1293907ab15053b7883aab248"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.642951 29700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:26.647697 29700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:26.648027 29700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:26.648233 29700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:26.653034 29700 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:32859
I20260812 05:25:26.653070 29769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:32859 every 8 connection(s)
I20260812 05:25:26.653486 29700 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:26.654284 29771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:26.657588 29771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce: Bootstrap starting.
I20260812 05:25:26.658221 29771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:26.658528 29771 log.cc:824] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:26.659314 29771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce: No bootstrap required, opened a new log
I20260812 05:25:26.661151 29771 raft_consensus.cc:348] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b114957c126549aca201a914a7f69cce" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32859 } }
I20260812 05:25:26.661312 29771 raft_consensus.cc:374] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:26.661339 29771 raft_consensus.cc:729] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b114957c126549aca201a914a7f69cce, State: Initialized, Role: FOLLOWER
I20260812 05:25:26.661513 29771 consensus_queue.cc:260] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b114957c126549aca201a914a7f69cce" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32859 } }
I20260812 05:25:26.661609 29771 raft_consensus.cc:388] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:26.661638 29771 raft_consensus.cc:482] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:26.661684 29771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:26.662266 29771 raft_consensus.cc:504] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b114957c126549aca201a914a7f69cce" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32859 } }
I20260812 05:25:26.662426 29771 leader_election.cc:302] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b114957c126549aca201a914a7f69cce; no voters:
I20260812 05:25:26.662635 29771 leader_election.cc:288] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:26.662691 29774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:26.662837 29774 raft_consensus.cc:686] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [term 1 LEADER]: Becoming Leader. State: Replica: b114957c126549aca201a914a7f69cce, State: Running, Role: LEADER
I20260812 05:25:26.662933 29774 consensus_queue.cc:237] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b114957c126549aca201a914a7f69cce" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32859 } }
I20260812 05:25:26.662971 29771 sys_catalog.cc:564] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:26.663380 29778 sys_catalog.cc:455] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [sys.catalog]: SysCatalogTable state changed. Reason: New leader b114957c126549aca201a914a7f69cce. Latest consensus state: current_term: 1 leader_uuid: "b114957c126549aca201a914a7f69cce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b114957c126549aca201a914a7f69cce" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32859 } } }
I20260812 05:25:26.663570 29778 sys_catalog.cc:458] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:26.663722 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 29700
I20260812 05:25:26.663700 29777 sys_catalog.cc:455] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b114957c126549aca201a914a7f69cce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b114957c126549aca201a914a7f69cce" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32859 } } }
I20260812 05:25:26.663782 29777 sys_catalog.cc:458] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:26.663818 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:26.664053 24841 external_mini_cluster.cc:1428] Setting key ef2e6c3d08dc0dc26e98a3163064a5ec
I20260812 05:25:26.664068 29780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:26.664706 29780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:26.667236 29780 catalog_manager.cc:1347] Generated new cluster ID: c1c57f789ec6494d8d7f290a3db312b7
I20260812 05:25:26.667237 29790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.664811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.24.66.126:32859)
0812 05:25:26.665082 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:26.665091 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:26.665378 (+ 287us) server_negotiation.cc:407] Connection header received
0812 05:25:26.665602 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:26.665611 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:26.665664 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:26.665830 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:26.666622 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:26.666626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:26.666626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:26.666631 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:26.666691 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:26.666707 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:26.666798 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:26.666940 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 05:25:26.667008 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":83,"threads_started":1}
I20260812 05:25:26.667328 29780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:26.689587 29780 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:26.689834 29780 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:26.696542 29780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce: Generated new TSK 0
I20260812 05:25:26.696914 29780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:25:26.703352 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:32859
--builtin_ntp_servers=127.24.66.84:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 05:25:26.824836 29798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:26.825223 29798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:26.825284 29798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:26.827705 29798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:26.827811 29798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:26.827865 29798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:26.829918 29798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:32859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:26.830276 29798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:26.830770 29798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:26.837877 29804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:26.837976 29803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:26.837898 29807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:26.838282 29798 server_base.cc:1034] running on GCE node
I20260812 05:25:26.838601 29798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:26.838953 29798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:26.840099 29798 hybrid_clock.cc:648] HybridClock initialized: now 1786512326840068 us; error 49 us; skew 500 ppm
I20260812 05:25:26.841835 29798 webserver.cc:460] Webserver started at http://127.24.66.65:44737/ using document root <none> and password file <none>
I20260812 05:25:26.842139 29798 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:26.842213 29798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:26.842401 29798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:26.844095 29798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "fccfadaa342d4be68afc74ab48e95684"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "28f9a8c487ddf271f26df7f1336a13af"
server_key_iv: "bf683466324008961d51601115156eab"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.844578 29798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "fccfadaa342d4be68afc74ab48e95684"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "28f9a8c487ddf271f26df7f1336a13af"
server_key_iv: "bf683466324008961d51601115156eab"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.846673 29798 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260812 05:25:26.847802 29813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:26.848158 29798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:26.848289 29798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "fccfadaa342d4be68afc74ab48e95684"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "28f9a8c487ddf271f26df7f1336a13af"
server_key_iv: "bf683466324008961d51601115156eab"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.848380 29798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:26.853547 29798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:26.853926 29798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:26.854108 29798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:26.854408 29798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:26.854815 29798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:26.854882 29798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:26.854930 29798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:26.854960 29798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:26.863479 29798 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:41739
I20260812 05:25:26.863559 29936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:41739 every 8 connection(s)
I20260812 05:25:26.864007 29798 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:26.867295 29790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.864847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:45769 (local address 127.24.66.126:32859)
0812 05:25:26.864918 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:26.864920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:26.865326 (+ 406us) server_negotiation.cc:407] Connection header received
0812 05:25:26.865702 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:26.865703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:26.865717 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:26.865787 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:26.867078 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:26.867080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:26.867080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:26.867085 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:26.867100 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:26.867104 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:26.867163 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:26.867223 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 05:25:26.867238 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:25:26.867506 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.864844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:32859 (local address 127.24.66.65:45769)
0812 05:25:26.865095 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:26.865130 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:26.865602 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:26.865849 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:26.865853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:26.865861 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:26.866928 (+ 1067us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:26.866929 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:26.866959 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:26.866989 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:26.867160 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:26.867163 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:26.867165 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:25:26.867217 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 05:25:26.867267 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":98,"threads_started":1}
I20260812 05:25:26.868037 29937 heartbeater.cc:347] Connected to a master server at 127.24.66.126:32859
I20260812 05:25:26.868234 29937 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:26.868412 29937 heartbeater.cc:511] Master 127.24.66.126:32859 requested a full tablet report, sending...
I20260812 05:25:26.868937 29732 ts_manager.cc:194] Registered new tserver with Master: fccfadaa342d4be68afc74ab48e95684 (127.24.66.65:41739)
I20260812 05:25:26.870605 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 29798
I20260812 05:25:26.870714 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:26.870930 24841 external_mini_cluster.cc:1428] Setting key 02d382eeadf7d85bd847dddb19403985
I20260812 05:25:26.871883 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:32859
--builtin_ntp_servers=127.24.66.84:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 05:25:26.957010 29941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:26.957219 29941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:26.957242 29941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:26.958508 29941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:26.958551 29941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:26.958600 29941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:26.959955 29941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:32859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:26.960213 29941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:26.960525 29941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:26.966034 29947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:26.966037 29949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:26.966070 29941 server_base.cc:1034] running on GCE node
W20260812 05:25:26.966063 29946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:26.966722 29941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:26.966992 29941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:26.968168 29941 hybrid_clock.cc:648] HybridClock initialized: now 1786512326968157 us; error 31 us; skew 500 ppm
I20260812 05:25:26.969547 29941 webserver.cc:460] Webserver started at http://127.24.66.66:39613/ using document root <none> and password file <none>
I20260812 05:25:26.969843 29941 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:26.969923 29941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:26.970081 29941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:26.971275 29941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "433bc252407a4f639270020a31275f1a"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "c1db97c8280fe6d6947298ab115187e8"
server_key_iv: "f7be14ccd4feeb7858d88cf103bfe996"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.971658 29941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "433bc252407a4f639270020a31275f1a"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "c1db97c8280fe6d6947298ab115187e8"
server_key_iv: "f7be14ccd4feeb7858d88cf103bfe996"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.973084 29941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:25:26.974009 29955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:26.974201 29941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:26.974308 29941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "433bc252407a4f639270020a31275f1a"
format_stamp: "Formatted at 2026-08-12 05:25:26 on dist-test-slave-zpfg"
server_key: "c1db97c8280fe6d6947298ab115187e8"
server_key_iv: "f7be14ccd4feeb7858d88cf103bfe996"
server_key_version: "encryptionkey@0"
I20260812 05:25:26.974445 29941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:26.978135 29941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:26.978507 29941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:26.978662 29941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:26.978891 29941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:26.979219 29941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:26.979337 29941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:26.979382 29941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:26.979410 29941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:26.985638 29941 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:35159
I20260812 05:25:26.985692 30080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:35159 every 8 connection(s)
I20260812 05:25:26.986109 29941 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:26.987439 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 29941
I20260812 05:25:26.987542 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:26.987725 24841 external_mini_cluster.cc:1428] Setting key ebf1bde20225ccfcbe58b2813b7badc2
I20260812 05:25:26.988853 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:32859
--builtin_ntp_servers=127.24.66.84:40173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 05:25:26.989688 29790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.987007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:51371 (local address 127.24.66.126:32859)
0812 05:25:26.987094 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:26.987097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:26.987348 (+ 251us) server_negotiation.cc:407] Connection header received
0812 05:25:26.987864 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:26.987866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:26.987873 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:26.987935 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:26.989422 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:26.989424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:26.989424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:26.989429 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:26.989445 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:26.989450 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:26.989500 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:26.989598 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 05:25:26.989613 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 05:25:26.989908 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:26.986907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:32859 (local address 127.24.66.66:51371)
0812 05:25:26.987110 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:26.987251 (+ 141us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:26.987746 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:26.988026 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:26.988029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:26.988037 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:26.989259 (+ 1222us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:26.989259 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:26.989289 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:26.989318 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:26.989503 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:26.989508 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:26.989512 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:26.989609 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 05:25:26.989681 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260812 05:25:26.990530 30081 heartbeater.cc:347] Connected to a master server at 127.24.66.126:32859
I20260812 05:25:26.990702 30081 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:26.990880 30081 heartbeater.cc:511] Master 127.24.66.126:32859 requested a full tablet report, sending...
I20260812 05:25:26.991283 29732 ts_manager.cc:194] Registered new tserver with Master: 433bc252407a4f639270020a31275f1a (127.24.66.66:35159)
W20260812 05:25:27.076081 30084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:27.076336 30084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:27.076362 30084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:27.077795 30084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:27.077865 30084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:27.077919 30084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:27.079439 30084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:32859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:27.079722 30084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:27.080035 30084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:27.085413 30090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:27.085422 30092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:27.085515 30084 server_base.cc:1034] running on GCE node
W20260812 05:25:27.085436 30089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:27.085865 30084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:27.086082 30084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:27.087224 30084 hybrid_clock.cc:648] HybridClock initialized: now 1786512327087203 us; error 36 us; skew 500 ppm
I20260812 05:25:27.088619 30084 webserver.cc:460] Webserver started at http://127.24.66.67:36901/ using document root <none> and password file <none>
I20260812 05:25:27.088898 30084 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:27.088950 30084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:27.089088 30084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:27.090191 30084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "f343f54e269b4ad682d7ff779a304e4b"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "c50fd4469afc50f9068029cb10d98640"
server_key_iv: "3b792b502d494881e1a245085c1e26d1"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.090627 30084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "f343f54e269b4ad682d7ff779a304e4b"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "c50fd4469afc50f9068029cb10d98640"
server_key_iv: "3b792b502d494881e1a245085c1e26d1"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.092028 30084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:27.092792 30099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:27.092974 30084 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:25:27.093077 30084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "f343f54e269b4ad682d7ff779a304e4b"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "c50fd4469afc50f9068029cb10d98640"
server_key_iv: "3b792b502d494881e1a245085c1e26d1"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.093161 30084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:27.096783 30084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:27.097059 30084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:27.097216 30084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:27.097437 30084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:27.097754 30084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:27.097811 30084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:27.097851 30084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:27.097879 30084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:27.103782 30084 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:39099
I20260812 05:25:27.103916 30217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:39099 every 8 connection(s)
I20260812 05:25:27.104236 30084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:27.105320 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 30084
I20260812 05:25:27.105438 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:27.105638 24841 external_mini_cluster.cc:1428] Setting key ef25fe6cb0d67ad32caa03e13af3ac6a
I20260812 05:25:27.107826 29790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.105134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:60531 (local address 127.24.66.126:32859)
0812 05:25:27.105222 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.105225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.105527 (+ 302us) server_negotiation.cc:407] Connection header received
0812 05:25:27.106393 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.106393 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.106399 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.106464 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:27.107619 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:27.107620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:27.107621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:27.107625 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:27.107640 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:27.107644 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:27.107705 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:27.107755 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 05:25:27.107769 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 05:25:27.108100 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.105065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:32859 (local address 127.24.66.67:60531)
0812 05:25:27.105384 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:27.105440 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:27.106135 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:27.106553 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:27.106557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:27.106568 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:27.107461 (+ 893us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:27.107462 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:27.107490 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:27.107518 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:27.107705 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:27.107707 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:27.107710 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:27.107808 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 05:25:27.107858 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":194,"threads_started":1}
I20260812 05:25:27.108573 30218 heartbeater.cc:347] Connected to a master server at 127.24.66.126:32859
I20260812 05:25:27.108661 30218 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:27.108779 30218 heartbeater.cc:511] Master 127.24.66.126:32859 requested a full tablet report, sending...
I20260812 05:25:27.109138 29732 ts_manager.cc:194] Registered new tserver with Master: f343f54e269b4ad682d7ff779a304e4b (127.24.66.67:39099)
I20260812 05:25:27.117460 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:25:27.120033 29790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.118950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48506 (local address 127.24.66.126:32859)
0812 05:25:27.119035 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.119038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.119138 (+ 100us) server_negotiation.cc:407] Connection header received
0812 05:25:27.119178 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.119178 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.119183 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.119233 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:27.119781 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:27.119782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:27.119782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:27.119786 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:27.119801 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:27.119805 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:27.119867 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:27.119899 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 05:25:27.119912 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 05:25:27.121883 29732 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:27.123719 29732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:27.127772 30226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.125957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:35159 (local address 127.0.0.1:57474)
0812 05:25:27.126232 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:27.126243 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:27.126292 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:27.126629 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:27.126630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:27.126634 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:27.127223 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:27.127224 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:27.127243 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:27.127267 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:27.127585 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:27.127593 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:27.127596 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:27.127661 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 05:25:27.127676 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":92,"threads_started":1}
I20260812 05:25:27.127830 30230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.126049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39100 (local address 127.24.66.67:39099)
0812 05:25:27.126399 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.126404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.126414 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:27.126441 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.126443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.126453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.126537 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:27.127516 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:27.127524 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:27.127525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:27.127530 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:27.127565 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:27.127592 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:27.127659 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:27.127726 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 05:25:27.127741 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":88,"threads_started":1}
I20260812 05:25:27.127847 30229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.125937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:39099 (local address 127.0.0.1:39100)
0812 05:25:27.126135 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:27.126182 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:27.126243 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:27.126547 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:27.126554 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:27.126560 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:27.127408 (+ 848us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:27.127409 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:27.127415 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:27.127425 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:27.127655 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:27.127656 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:27.127657 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 05:25:27.127716 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 05:25:27.127731 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":80,"threads_started":1}
I20260812 05:25:27.128118 30231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.126068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57474 (local address 127.24.66.66:35159)
0812 05:25:27.126462 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.126466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.126477 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:27.126507 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.126509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.126519 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.126808 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:27.127358 (+ 550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:27.127368 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:27.127369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:27.127374 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:27.127417 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:27.127438 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:27.127560 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:27.128016 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 05:25:27.128035 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":98,"threads_started":1}
I20260812 05:25:27.128662 30232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.126165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:41739 (local address 127.0.0.1:33018)
0812 05:25:27.126312 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:27.126323 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:27.126384 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:27.127410 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:27.127411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:27.127413 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:27.128248 (+ 835us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:27.128248 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:27.128256 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:27.128270 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:27.128523 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:27.128523 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:27.128524 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 05:25:27.128591 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:25:27.128604 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":135,"threads_started":1}
I20260812 05:25:27.128580 30148 tablet_service.cc:1467] Processing CreateTablet for tablet b1ec2f400f984a7389bcda7c3ad9a1da (DEFAULT_TABLE table=test-table [id=5a8cb43f8a494c178d78fefb18fc5c51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:27.128912 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.126485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33018 (local address 127.24.66.65:41739)
0812 05:25:27.126686 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.126690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.126700 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:27.126726 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.126729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.126737 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.127001 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:27.128368 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:27.128379 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:27.128379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:27.128383 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:27.128418 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:27.128452 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:27.128515 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:27.128621 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 05:25:27.128638 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
I20260812 05:25:27.129060 30148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ec2f400f984a7389bcda7c3ad9a1da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:27.128985 30008 tablet_service.cc:1467] Processing CreateTablet for tablet b1ec2f400f984a7389bcda7c3ad9a1da (DEFAULT_TABLE table=test-table [id=5a8cb43f8a494c178d78fefb18fc5c51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:27.129420 30008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ec2f400f984a7389bcda7c3ad9a1da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:27.129733 29865 tablet_service.cc:1467] Processing CreateTablet for tablet b1ec2f400f984a7389bcda7c3ad9a1da (DEFAULT_TABLE table=test-table [id=5a8cb43f8a494c178d78fefb18fc5c51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:27.130071 29865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ec2f400f984a7389bcda7c3ad9a1da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:27.132319 30235 tablet_bootstrap.cc:492] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a: Bootstrap starting.
I20260812 05:25:27.132664 30237 tablet_bootstrap.cc:492] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b: Bootstrap starting.
I20260812 05:25:27.133229 30235 tablet_bootstrap.cc:654] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:27.133325 30237 tablet_bootstrap.cc:654] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:27.133671 30237 log.cc:824] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:27.133667 30235 log.cc:824] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:27.134069 30238 tablet_bootstrap.cc:492] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684: Bootstrap starting.
I20260812 05:25:27.134536 30237 tablet_bootstrap.cc:492] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b: No bootstrap required, opened a new log
I20260812 05:25:27.134622 30237 ts_tablet_manager.cc:1397] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:25:27.134616 30235 tablet_bootstrap.cc:492] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a: No bootstrap required, opened a new log
I20260812 05:25:27.134786 30235 ts_tablet_manager.cc:1397] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:25:27.134806 30238 tablet_bootstrap.cc:654] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:27.135130 30238 log.cc:824] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:27.135968 30238 tablet_bootstrap.cc:492] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684: No bootstrap required, opened a new log
I20260812 05:25:27.136039 30238 ts_tablet_manager.cc:1397] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:25:27.137054 30237 raft_consensus.cc:348] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f343f54e269b4ad682d7ff779a304e4b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39099 } } peers { permanent_uuid: "fccfadaa342d4be68afc74ab48e95684" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41739 } } peers { permanent_uuid: "433bc252407a4f639270020a31275f1a" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35159 } }
I20260812 05:25:27.137238 30237 raft_consensus.cc:374] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:27.137287 30237 raft_consensus.cc:729] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f343f54e269b4ad682d7ff779a304e4b, State: Initialized, Role: FOLLOWER
I20260812 05:25:27.137415 30237 consensus_queue.cc:260] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f343f54e269b4ad682d7ff779a304e4b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39099 } } peers { permanent_uuid: "fccfadaa342d4be68afc74ab48e95684" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41739 } } peers { permanent_uuid: "433bc252407a4f639270020a31275f1a" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35159 } }
I20260812 05:25:27.137740 30218 heartbeater.cc:503] Master 127.24.66.126:32859 was elected leader, sending a full tablet report...
I20260812 05:25:27.137737 30237 ts_tablet_manager.cc:1428] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:25:27.137849 30235 raft_consensus.cc:348] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f343f54e269b4ad682d7ff779a304e4b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39099 } } peers { permanent_uuid: "fccfadaa342d4be68afc74ab48e95684" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41739 } } peers { permanent_uuid: "433bc252407a4f639270020a31275f1a" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35159 } }
I20260812 05:25:27.138003 30235 raft_consensus.cc:374] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:27.138124 30235 raft_consensus.cc:729] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433bc252407a4f639270020a31275f1a, State: Initialized, Role: FOLLOWER
I20260812 05:25:27.138370 30235 consensus_queue.cc:260] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f343f54e269b4ad682d7ff779a304e4b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39099 } } peers { permanent_uuid: "fccfadaa342d4be68afc74ab48e95684" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41739 } } peers { permanent_uuid: "433bc252407a4f639270020a31275f1a" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35159 } }
I20260812 05:25:27.138742 30235 ts_tablet_manager.cc:1428] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 05:25:27.138912 30081 heartbeater.cc:503] Master 127.24.66.126:32859 was elected leader, sending a full tablet report...
I20260812 05:25:27.139024 30238 raft_consensus.cc:348] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f343f54e269b4ad682d7ff779a304e4b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39099 } } peers { permanent_uuid: "fccfadaa342d4be68afc74ab48e95684" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41739 } } peers { permanent_uuid: "433bc252407a4f639270020a31275f1a" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35159 } }
I20260812 05:25:27.139190 30238 raft_consensus.cc:374] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:27.139233 30238 raft_consensus.cc:729] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fccfadaa342d4be68afc74ab48e95684, State: Initialized, Role: FOLLOWER
I20260812 05:25:27.139346 30238 consensus_queue.cc:260] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f343f54e269b4ad682d7ff779a304e4b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39099 } } peers { permanent_uuid: "fccfadaa342d4be68afc74ab48e95684" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41739 } } peers { permanent_uuid: "433bc252407a4f639270020a31275f1a" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35159 } }
I20260812 05:25:27.139657 29937 heartbeater.cc:503] Master 127.24.66.126:32859 was elected leader, sending a full tablet report...
I20260812 05:25:27.139885 30238 ts_tablet_manager.cc:1428] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260812 05:25:27.172731 30246 raft_consensus.cc:482] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:27.172878 30246 raft_consensus.cc:504] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f343f54e269b4ad682d7ff779a304e4b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39099 } } peers { permanent_uuid: "fccfadaa342d4be68afc74ab48e95684" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41739 } } peers { permanent_uuid: "433bc252407a4f639270020a31275f1a" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35159 } }
I20260812 05:25:27.173260 30246 leader_election.cc:288] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f343f54e269b4ad682d7ff779a304e4b (127.24.66.67:39099), fccfadaa342d4be68afc74ab48e95684 (127.24.66.65:41739)
I20260812 05:25:27.174799 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.173465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:39099 (local address 127.24.66.66:58223)
0812 05:25:27.173530 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:27.173542 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:27.173584 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:27.173869 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:27.173871 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:27.173874 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:27.174415 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:27.174416 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:27.174426 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:27.174442 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:27.174641 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:27.174642 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:27.174643 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 05:25:27.174714 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 05:25:27.174730 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 05:25:27.174842 30230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.173580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:58223 (local address 127.24.66.67:39099)
0812 05:25:27.173657 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.173659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.173667 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:27.173697 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.173698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.173702 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.173861 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:27.174535 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:27.174536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:27.174537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:27.174541 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:27.174561 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:27.174566 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:27.174633 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:27.174721 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 05:25:27.174738 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 05:25:27.174989 30234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.173791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:34919 (local address 127.24.66.65:41739)
0812 05:25:27.173884 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.173886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.173896 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:27.173961 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.173962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.173967 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.174027 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:27.174700 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:27.174701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:27.174701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:27.174706 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:27.174722 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:27.174726 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:27.174773 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:27.174872 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 05:25:27.174889 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:25:27.175170 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.173600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:41739 (local address 127.24.66.66:34919)
0812 05:25:27.173785 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:27.173794 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:27.173904 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:27.174032 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:27.174033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:27.174035 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:27.174614 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:27.174614 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:27.174621 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:27.174634 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:27.174778 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:27.174779 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:27.174779 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 05:25:27.175081 (+ 302us) client_negotiation.cc:240] Negotiation successful
0812 05:25:27.175096 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":108,"threads_started":1}
I20260812 05:25:27.175165 30169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ec2f400f984a7389bcda7c3ad9a1da" candidate_uuid: "433bc252407a4f639270020a31275f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f343f54e269b4ad682d7ff779a304e4b" is_pre_election: true
I20260812 05:25:27.175359 30169 raft_consensus.cc:2454] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 433bc252407a4f639270020a31275f1a in term 0.
I20260812 05:25:27.175534 29886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ec2f400f984a7389bcda7c3ad9a1da" candidate_uuid: "433bc252407a4f639270020a31275f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fccfadaa342d4be68afc74ab48e95684" is_pre_election: true
I20260812 05:25:27.175607 29958 leader_election.cc:302] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 433bc252407a4f639270020a31275f1a, f343f54e269b4ad682d7ff779a304e4b; no voters:
I20260812 05:25:27.175717 29886 raft_consensus.cc:2454] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 433bc252407a4f639270020a31275f1a in term 0.
I20260812 05:25:27.175738 30246 raft_consensus.cc:2781] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:27.175817 30246 raft_consensus.cc:482] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:27.175858 30246 raft_consensus.cc:3037] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:27.176697 30246 raft_consensus.cc:504] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f343f54e269b4ad682d7ff779a304e4b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39099 } } peers { permanent_uuid: "fccfadaa342d4be68afc74ab48e95684" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41739 } } peers { permanent_uuid: "433bc252407a4f639270020a31275f1a" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35159 } }
I20260812 05:25:27.176846 30246 leader_election.cc:288] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [CANDIDATE]: Term 1 election: Requested vote from peers f343f54e269b4ad682d7ff779a304e4b (127.24.66.67:39099), fccfadaa342d4be68afc74ab48e95684 (127.24.66.65:41739)
I20260812 05:25:27.177022 30169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ec2f400f984a7389bcda7c3ad9a1da" candidate_uuid: "433bc252407a4f639270020a31275f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f343f54e269b4ad682d7ff779a304e4b"
I20260812 05:25:27.177073 29886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ec2f400f984a7389bcda7c3ad9a1da" candidate_uuid: "433bc252407a4f639270020a31275f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fccfadaa342d4be68afc74ab48e95684"
I20260812 05:25:27.177134 30169 raft_consensus.cc:3037] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:27.177136 29886 raft_consensus.cc:3037] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:27.177977 29886 raft_consensus.cc:2454] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 433bc252407a4f639270020a31275f1a in term 1.
I20260812 05:25:27.178180 30169 raft_consensus.cc:2454] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 433bc252407a4f639270020a31275f1a in term 1.
I20260812 05:25:27.178311 29958 leader_election.cc:302] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 433bc252407a4f639270020a31275f1a, fccfadaa342d4be68afc74ab48e95684; no voters:
I20260812 05:25:27.178503 30246 raft_consensus.cc:2781] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:27.178668 30246 raft_consensus.cc:686] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 1 LEADER]: Becoming Leader. State: Replica: 433bc252407a4f639270020a31275f1a, State: Running, Role: LEADER
I20260812 05:25:27.178781 30246 consensus_queue.cc:237] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f343f54e269b4ad682d7ff779a304e4b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39099 } } peers { permanent_uuid: "fccfadaa342d4be68afc74ab48e95684" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41739 } } peers { permanent_uuid: "433bc252407a4f639270020a31275f1a" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35159 } }
I20260812 05:25:27.179714 29731 catalog_manager.cc:5495] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 433bc252407a4f639270020a31275f1a (127.24.66.66). New cstate: current_term: 1 leader_uuid: "433bc252407a4f639270020a31275f1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f343f54e269b4ad682d7ff779a304e4b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fccfadaa342d4be68afc74ab48e95684" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "433bc252407a4f639270020a31275f1a" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35159 } health_report { overall_health: HEALTHY } } }
I20260812 05:25:27.189021 30231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.188006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57488 (local address 127.24.66.66:35159)
0812 05:25:27.188068 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.188069 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.188076 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:25:27.188109 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.188110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.188121 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.188188 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:27.188735 (+ 547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:27.188737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:27.188737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:27.188742 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:27.188758 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:27.188762 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:27.188843 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:27.188929 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 05:25:27.188944 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:25:27.191083 30169 raft_consensus.cc:1261] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b [term 1 FOLLOWER]: Refusing update from remote peer 433bc252407a4f639270020a31275f1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:27.191084 29886 raft_consensus.cc:1261] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684 [term 1 FOLLOWER]: Refusing update from remote peer 433bc252407a4f639270020a31275f1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:27.191479 30246 consensus_queue.cc:1035] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f343f54e269b4ad682d7ff779a304e4b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:27.191632 30249 consensus_queue.cc:1035] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fccfadaa342d4be68afc74ab48e95684" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:27.193989 30252 mvcc.cc:205] Tried to move back new op lower bound from 7317554492172005376 to 7317554492125102080. Current Snapshot: MvccSnapshot[applied={T|T < 7317554492172005376}]
I20260812 05:25:27.194578 30254 mvcc.cc:205] Tried to move back new op lower bound from 7317554492172005376 to 7317554492125102080. Current Snapshot: MvccSnapshot[applied={T|T < 7317554492172005376}]
I20260812 05:25:27.196563 29732 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:27.196662 29732 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:27.197420 29732 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b114957c126549aca201a914a7f69cce: Sending DeleteTablet for 3 replicas of tablet b1ec2f400f984a7389bcda7c3ad9a1da
I20260812 05:25:27.197743 30148 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ec2f400f984a7389bcda7c3ad9a1da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:27 UTC) from {username='slave'} at 127.0.0.1:39100
I20260812 05:25:27.197744 30008 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ec2f400f984a7389bcda7c3ad9a1da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:27 UTC) from {username='slave'} at 127.0.0.1:57474
I20260812 05:25:27.197810 29865 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ec2f400f984a7389bcda7c3ad9a1da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:27 UTC) from {username='slave'} at 127.0.0.1:33018
I20260812 05:25:27.198040 30263 tablet_replica.cc:330] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684: stopping tablet replica
I20260812 05:25:27.198158 30263 raft_consensus.cc:2229] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:27.198264 30263 raft_consensus.cc:2258] T b1ec2f400f984a7389bcda7c3ad9a1da P fccfadaa342d4be68afc74ab48e95684 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:27.198262 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 29798
I20260812 05:25:27.203907 30261 tablet_replica.cc:330] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a: stopping tablet replica
I20260812 05:25:27.204077 30261 raft_consensus.cc:2229] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:27.204202 30261 raft_consensus.cc:2258] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:27.204509 30262 tablet_replica.cc:330] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b: stopping tablet replica
I20260812 05:25:27.204670 30262 raft_consensus.cc:2229] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:27.204823 30262 raft_consensus.cc:2258] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:27.205262 30261 ts_tablet_manager.cc:1905] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:27.205470 30262 ts_tablet_manager.cc:1905] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:27.207414 30262 ts_tablet_manager.cc:1918] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:27.207412 30261 ts_tablet_manager.cc:1918] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:27.207509 30261 log.cc:1196] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/b1ec2f400f984a7389bcda7c3ad9a1da
I20260812 05:25:27.207654 30262 log.cc:1196] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/b1ec2f400f984a7389bcda7c3ad9a1da
I20260812 05:25:27.207861 30261 ts_tablet_manager.cc:1939] T b1ec2f400f984a7389bcda7c3ad9a1da P 433bc252407a4f639270020a31275f1a: Deleting consensus metadata
I20260812 05:25:27.208117 30262 ts_tablet_manager.cc:1939] T b1ec2f400f984a7389bcda7c3ad9a1da P f343f54e269b4ad682d7ff779a304e4b: Deleting consensus metadata
I20260812 05:25:27.208438 29717 catalog_manager.cc:4841] TS 433bc252407a4f639270020a31275f1a (127.24.66.66:35159): tablet b1ec2f400f984a7389bcda7c3ad9a1da (table test-table [id=5a8cb43f8a494c178d78fefb18fc5c51]) successfully deleted
I20260812 05:25:27.208643 29718 catalog_manager.cc:4841] TS f343f54e269b4ad682d7ff779a304e4b (127.24.66.67:39099): tablet b1ec2f400f984a7389bcda7c3ad9a1da (table test-table [id=5a8cb43f8a494c178d78fefb18fc5c51]) successfully deleted
W20260812 05:25:27.209893 29718 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:41739 (error 108)
I20260812 05:25:27.210239 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 29941
I20260812 05:25:27.212287 30232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.212119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:41739 (local address 127.0.0.1:33026)
0812 05:25:27.212190 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:27.212244 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:41739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 05:25:27.212527 29718 catalog_manager.cc:4584] TS fccfadaa342d4be68afc74ab48e95684 (127.24.66.65:41739): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1ec2f400f984a7389bcda7c3ad9a1da: Network error: Client connection negotiation failed: client connection to 127.24.66.65:41739: connect: Connection refused (error 111)
I20260812 05:25:27.219535 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 30084
I20260812 05:25:27.224607 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 29700
2026-08-12T05:25:27Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (715 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 05:25:27.245642 24841 mini_oidc.cc:152] Starting JWKS server
I20260812 05:25:27.247181 24841 webserver.cc:460] Webserver started at https://127.0.0.1:45063/ using document root <none> and password file <none>
I20260812 05:25:27.247270 24841 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 05:25:27.247481 24841 webserver.cc:460] Webserver started at http://127.0.0.1:44709/ using document root <none> and password file <none>
2026-08-12T05:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:27Z Disabled control of system clock
I20260812 05:25:27.257395 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:45727
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:35893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:45727
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45063/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/testchainca.cert with env {}
W20260812 05:25:27.343559 30270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:27.343786 30270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:27.343809 30270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:27.344936 30270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:27.344976 30270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:27.344990 30270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:27.345001 30270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:27.345013 30270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 05:25:27.345026 30270 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 05:25:27.345037 30270 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45063/jwks.json
I20260812 05:25:27.346683 30270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:45727
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:45727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45063/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:27.346903 30270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:27.347087 30270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:27.352371 30275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:27.352487 30270 server_base.cc:1034] running on GCE node
W20260812 05:25:27.352383 30276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:27.352383 30278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:27.352967 30270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:27.353293 30270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:27.354475 30270 hybrid_clock.cc:648] HybridClock initialized: now 1786512327354461 us; error 30 us; skew 500 ppm
I20260812 05:25:27.355926 30270 webserver.cc:460] Webserver started at http://127.24.66.126:43893/ using document root <none> and password file <none>
I20260812 05:25:27.356331 30270 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:27.356391 30270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:27.356599 30270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:27.357806 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "f97ef77772f44ca88a6daca74b0fabcf"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "196837b3c9344eb73a0c9d529d97a770"
server_key_iv: "ca841b3360205ab428bc3e72d79a26ff"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.358232 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "f97ef77772f44ca88a6daca74b0fabcf"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "196837b3c9344eb73a0c9d529d97a770"
server_key_iv: "ca841b3360205ab428bc3e72d79a26ff"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.359833 30270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:27.360642 30284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:27.360852 30270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:27.360945 30270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "f97ef77772f44ca88a6daca74b0fabcf"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "196837b3c9344eb73a0c9d529d97a770"
server_key_iv: "ca841b3360205ab428bc3e72d79a26ff"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.361022 30270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:27.379127 30270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:27.379492 30270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:27.379657 30270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:27.384271 30270 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:45727
I20260812 05:25:27.384413 30338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:45727 every 8 connection(s)
I20260812 05:25:27.384734 30270 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:27.385833 30339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:27.388406 30339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf: Bootstrap starting.
I20260812 05:25:27.388994 30339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:27.389254 30339 log.cc:824] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:27.390096 30339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf: No bootstrap required, opened a new log
I20260812 05:25:27.391994 30339 raft_consensus.cc:348] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97ef77772f44ca88a6daca74b0fabcf" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45727 } }
I20260812 05:25:27.392158 30339 raft_consensus.cc:374] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:27.392185 30339 raft_consensus.cc:729] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97ef77772f44ca88a6daca74b0fabcf, State: Initialized, Role: FOLLOWER
I20260812 05:25:27.392306 30339 consensus_queue.cc:260] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97ef77772f44ca88a6daca74b0fabcf" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45727 } }
I20260812 05:25:27.392409 30339 raft_consensus.cc:388] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:27.392454 30339 raft_consensus.cc:482] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:27.392513 30339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:27.392995 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 30270
I20260812 05:25:27.393090 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:27.393189 30339 raft_consensus.cc:504] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97ef77772f44ca88a6daca74b0fabcf" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45727 } }
I20260812 05:25:27.393285 24841 external_mini_cluster.cc:1428] Setting key 33421d99e31e649d1026b778b7bd8d5a
I20260812 05:25:27.393326 30339 leader_election.cc:302] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f97ef77772f44ca88a6daca74b0fabcf; no voters:
I20260812 05:25:27.393564 30339 leader_election.cc:288] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:27.393640 30342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:27.393812 30342 raft_consensus.cc:686] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [term 1 LEADER]: Becoming Leader. State: Replica: f97ef77772f44ca88a6daca74b0fabcf, State: Running, Role: LEADER
I20260812 05:25:27.393885 30339 sys_catalog.cc:564] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:27.393929 30342 consensus_queue.cc:237] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97ef77772f44ca88a6daca74b0fabcf" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45727 } }
I20260812 05:25:27.394404 30343 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f97ef77772f44ca88a6daca74b0fabcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97ef77772f44ca88a6daca74b0fabcf" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45727 } } }
I20260812 05:25:27.394492 30344 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader f97ef77772f44ca88a6daca74b0fabcf. Latest consensus state: current_term: 1 leader_uuid: "f97ef77772f44ca88a6daca74b0fabcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97ef77772f44ca88a6daca74b0fabcf" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45727 } } }
I20260812 05:25:27.394508 30343 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:27.394645 30344 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:27.394851 30353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:27.395385 30353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:27.397421 30361 mvcc.cc:205] Tried to move back new op lower bound from 7317554493016514560 to 7317554493005955072. Current Snapshot: MvccSnapshot[applied={T|T < 7317554493016514560}]
I20260812 05:25:27.397796 30353 catalog_manager.cc:1347] Generated new cluster ID: 1549595c27e54a66bc08e31823a816b0
I20260812 05:25:27.397838 30353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:27.400481 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.394074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49102 (local address 127.24.66.126:45727)
0812 05:25:27.394853 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.394863 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.394900 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:25:27.395021 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.395028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.395116 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.395269 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:27.395522 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.396804 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.398030 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.398670 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.398791 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:27.399715 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:27.399719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:27.399720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:27.399940 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:27.399983 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:27.399998 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:27.400062 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:27.400176 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 05:25:27.400262 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":77,"threads_started":1}
I20260812 05:25:27.401542 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:25:27.406198 30353 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:27.406765 30353 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:27.418390 30353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f97ef77772f44ca88a6daca74b0fabcf: Generated new TSK 0
I20260812 05:25:27.418620 30353 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 05:25:27.442888 30369 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 05:25:27.443068 30369 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 05:25:27.443341 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.427257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.24.66.126:45727)
0812 05:25:27.427314 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.427316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.427414 (+ 98us) server_negotiation.cc:407] Connection header received
0812 05:25:27.427544 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.427545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.427555 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.427625 (+ 70us) server_negotiation.cc:226] Negotiated authn=JWT
0812 05:25:27.427841 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.428310 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.429599 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.429783 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.429835 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:27.430100 (+ 265us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 05:25:27.443059 (+ 12959us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45063/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 05:25:27.443258 (+ 199us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49114: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45063/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 05:25:27.443418 30346 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49114: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45063/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 05:25:27.443977 30368 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:27.427121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45727 (local address 127.0.0.1:49114)
0812 05:25:27.427345 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:27.427358 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:27.427439 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:27.427662 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:27.427663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:27.427665 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 05:25:27.427737 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:27.427739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.428429 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.428430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:27.429503 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:27.429507 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.429875 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.429876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:27.429949 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:27.429976 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 05:25:27.443318 (+ 13342us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45063/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 05:25:27.443484 (+ 166us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.66.126:45727: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45063/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":111,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.66.126:45727: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45063/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 05:25:27.444617 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 30270
2026-08-12T05:25:27Z chronyd exiting
I20260812 05:25:27.483950 24841 test_util.cc:175] -----------------------------------------------
I20260812 05:25:27.484038 24841 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786512301961891-24841-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (239 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 05:25:27.484974 24841 mini_oidc.cc:152] Starting JWKS server
I20260812 05:25:27.485738 24841 webserver.cc:460] Webserver started at https://127.0.0.1:36411/ using document root <none> and password file <none>
I20260812 05:25:27.485822 24841 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 05:25:27.485983 24841 webserver.cc:460] Webserver started at http://127.0.0.1:44979/ using document root <none> and password file <none>
2026-08-12T05:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:27Z Disabled control of system clock
I20260812 05:25:27.494966 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39969
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:37849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:39969
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36411/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/testchainca.cert with env {}
W20260812 05:25:27.580459 30376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:27.580734 30376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:27.580765 30376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:27.582005 30376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:27.582049 30376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:27.582062 30376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:27.582077 30376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:27.582088 30376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 05:25:27.582101 30376 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 05:25:27.582113 30376 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36411/jwks.json
I20260812 05:25:27.583709 30376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:37849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:39969
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36411/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:27.583923 30376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:27.584100 30376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:25:27.589551 30376 server_base.cc:1034] running on GCE node
W20260812 05:25:27.589541 30384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:27.589502 30382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:27.589514 30381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:27.590027 30376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:27.590404 30376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:27.591580 30376 hybrid_clock.cc:648] HybridClock initialized: now 1786512327591556 us; error 36 us; skew 500 ppm
I20260812 05:25:27.593072 30376 webserver.cc:460] Webserver started at http://127.24.66.126:40019/ using document root <none> and password file <none>
I20260812 05:25:27.593366 30376 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:27.593446 30376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:27.593602 30376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:27.594832 30376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "7000e819f86542edb13944930b92d73b"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "72e9b718e0b92ca137579cdd56d06b55"
server_key_iv: "771087c077bb5690e138ca01784c3864"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.595216 30376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "7000e819f86542edb13944930b92d73b"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "72e9b718e0b92ca137579cdd56d06b55"
server_key_iv: "771087c077bb5690e138ca01784c3864"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.596719 30376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:27.597683 30391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:27.597899 30376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:27.598021 30376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "7000e819f86542edb13944930b92d73b"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "72e9b718e0b92ca137579cdd56d06b55"
server_key_iv: "771087c077bb5690e138ca01784c3864"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.598107 30376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:27.609587 30376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:27.609994 30376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:27.610188 30376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:27.614779 30376 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:39969
I20260812 05:25:27.614820 30449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:39969 every 8 connection(s)
I20260812 05:25:27.615238 30376 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:27.616037 30450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:27.619119 30450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b: Bootstrap starting.
I20260812 05:25:27.619823 30450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:27.619819 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 30376
I20260812 05:25:27.619922 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:27.620150 24841 external_mini_cluster.cc:1428] Setting key 58c39d32ca93068b1d7db6f77cfa417f
I20260812 05:25:27.620186 30450 log.cc:824] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:27.621009 30450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b: No bootstrap required, opened a new log
I20260812 05:25:27.624105 30450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7000e819f86542edb13944930b92d73b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39969 } }
I20260812 05:25:27.624331 30450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:27.624387 30450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7000e819f86542edb13944930b92d73b, State: Initialized, Role: FOLLOWER
I20260812 05:25:27.624555 30450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7000e819f86542edb13944930b92d73b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39969 } }
I20260812 05:25:27.624641 30450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:27.624670 30450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:27.624735 30450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:27.625593 30450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7000e819f86542edb13944930b92d73b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39969 } }
I20260812 05:25:27.625731 30450 leader_election.cc:302] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7000e819f86542edb13944930b92d73b; no voters:
I20260812 05:25:27.625960 30450 leader_election.cc:288] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:27.626052 30455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:27.626235 30455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [term 1 LEADER]: Becoming Leader. State: Replica: 7000e819f86542edb13944930b92d73b, State: Running, Role: LEADER
I20260812 05:25:27.626377 30455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7000e819f86542edb13944930b92d73b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39969 } }
I20260812 05:25:27.626379 30450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:27.626917 30456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7000e819f86542edb13944930b92d73b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7000e819f86542edb13944930b92d73b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39969 } } }
I20260812 05:25:27.627027 30456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:27.627082 30457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7000e819f86542edb13944930b92d73b. Latest consensus state: current_term: 1 leader_uuid: "7000e819f86542edb13944930b92d73b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7000e819f86542edb13944930b92d73b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39969 } } }
I20260812 05:25:27.627177 30457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:27.627040 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.620725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43678 (local address 127.24.66.126:39969)
0812 05:25:27.621006 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.621015 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.621068 (+ 53us) server_negotiation.cc:407] Connection header received
0812 05:25:27.621551 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.621558 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.621621 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.621742 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:27.622049 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.623079 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.624436 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.625072 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.625204 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:27.626107 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:27.626111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:27.626111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:27.626405 (+ 294us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:27.626458 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:27.626485 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:27.626562 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:27.626685 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 05:25:27.626784 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":150,"threads_started":1}
I20260812 05:25:27.627794 30459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:27.628386 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:25:27.628594 30459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:27.630836 30459 catalog_manager.cc:1347] Generated new cluster ID: c22d9a342c424d2e86eecb17c01417e9
I20260812 05:25:27.630905 30459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:27.642324 30459 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:27.643185 30459 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:27.647775 30459 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7000e819f86542edb13944930b92d73b: Generated new TSK 0
I20260812 05:25:27.648021 30459 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 05:25:27.671446 30480 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 05:25:27.671561 30480 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 05:25:27.671924 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.655970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.24.66.126:39969)
0812 05:25:27.656038 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.656040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.656262 (+ 222us) server_negotiation.cc:407] Connection header received
0812 05:25:27.656336 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.656338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.656352 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.656448 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0812 05:25:27.656673 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.657792 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.658707 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.659035 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.659112 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:27.659341 (+ 229us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 05:25:27.671658 (+ 12317us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36411/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 05:25:27.671847 (+ 189us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43684: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36411/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 05:25:27.671993 30454 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43684: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36411/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 05:25:27.672029 30479 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:27.655846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39969 (local address 127.0.0.1:43684)
0812 05:25:27.656071 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:27.656084 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:27.656266 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:27.656521 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:27.656522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:27.656525 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0812 05:25:27.656589 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:27.656591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.657908 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.657909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:27.658626 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:27.658629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.659121 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.659122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:27.659168 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:27.659173 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 05:25:27.671794 (+ 12621us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36411/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 05:25:27.671922 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.66.126:39969: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36411/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":145,"threads_started":1}
I20260812 05:25:27.672487 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 30376
2026-08-12T05:25:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 05:25:27.712957 24841 mini_oidc.cc:152] Starting JWKS server
I20260812 05:25:27.713699 24841 webserver.cc:460] Webserver started at https://127.0.0.1:38977/ using document root <none> and password file <none>
I20260812 05:25:27.713742 24841 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 05:25:27.713941 24841 webserver.cc:460] Webserver started at http://127.0.0.1:33755/ using document root <none> and password file <none>
2026-08-12T05:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:27Z Disabled control of system clock
I20260812 05:25:27.723124 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:33987
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:42863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:33987
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38977/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 05:25:27.806891 30486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:27.807123 30486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:27.807147 30486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:27.808490 30486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:27.808542 30486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:27.808554 30486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:27.808568 30486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:27.808580 30486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 05:25:27.808636 30486 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 05:25:27.808663 30486 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38977/jwks.json
I20260812 05:25:27.810086 30486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:42863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:33987
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:33987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38977/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:27.810388 30486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:27.810708 30486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:27.816143 30495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:27.816211 30492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:27.816211 30491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:27.816435 30486 server_base.cc:1034] running on GCE node
I20260812 05:25:27.816699 30486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:27.817008 30486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:27.818157 30486 hybrid_clock.cc:648] HybridClock initialized: now 1786512327818133 us; error 38 us; skew 500 ppm
I20260812 05:25:27.819597 30486 webserver.cc:460] Webserver started at http://127.24.66.126:37931/ using document root <none> and password file <none>
I20260812 05:25:27.819864 30486 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:27.819939 30486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:27.820096 30486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:27.821233 30486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "48ff29444d0e4c838ca2c55ca2220838"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "4293569df4b0c7e61cf2a4d2d3594772"
server_key_iv: "b6d4a90643d25efb53b652ef305f8104"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.821630 30486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "48ff29444d0e4c838ca2c55ca2220838"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "4293569df4b0c7e61cf2a4d2d3594772"
server_key_iv: "b6d4a90643d25efb53b652ef305f8104"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.823138 30486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:27.824039 30501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:27.824288 30486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:27.824404 30486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "48ff29444d0e4c838ca2c55ca2220838"
format_stamp: "Formatted at 2026-08-12 05:25:27 on dist-test-slave-zpfg"
server_key: "4293569df4b0c7e61cf2a4d2d3594772"
server_key_iv: "b6d4a90643d25efb53b652ef305f8104"
server_key_version: "encryptionkey@0"
I20260812 05:25:27.824482 30486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:27.838608 30486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:27.838963 30486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:27.839129 30486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:27.843685 30486 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:33987
I20260812 05:25:27.843748 30556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:33987 every 8 connection(s)
I20260812 05:25:27.844161 30486 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:27.844925 30557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:27.847957 30557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838: Bootstrap starting.
I20260812 05:25:27.847985 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 30486
I20260812 05:25:27.848100 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:27.848287 24841 external_mini_cluster.cc:1428] Setting key 68b97cb7de9aedcc36d88ef8f9736d58
I20260812 05:25:27.848707 30557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:27.849103 30557 log.cc:824] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:27.849975 30557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838: No bootstrap required, opened a new log
I20260812 05:25:27.852322 30557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff29444d0e4c838ca2c55ca2220838" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33987 } }
I20260812 05:25:27.852522 30557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:27.852571 30557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48ff29444d0e4c838ca2c55ca2220838, State: Initialized, Role: FOLLOWER
I20260812 05:25:27.852728 30557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff29444d0e4c838ca2c55ca2220838" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33987 } }
I20260812 05:25:27.852803 30557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:27.852856 30557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:27.852907 30557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:27.853713 30557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff29444d0e4c838ca2c55ca2220838" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33987 } }
I20260812 05:25:27.853852 30557 leader_election.cc:302] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48ff29444d0e4c838ca2c55ca2220838; no voters:
I20260812 05:25:27.854074 30557 leader_election.cc:288] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:27.854254 30563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:27.854409 30557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:27.854413 30563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [term 1 LEADER]: Becoming Leader. State: Replica: 48ff29444d0e4c838ca2c55ca2220838, State: Running, Role: LEADER
I20260812 05:25:27.854537 30563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff29444d0e4c838ca2c55ca2220838" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33987 } }
I20260812 05:25:27.855084 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.848895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57842 (local address 127.24.66.126:33987)
0812 05:25:27.849095 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.849104 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.849130 (+ 26us) server_negotiation.cc:407] Connection header received
0812 05:25:27.849829 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.849838 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.849904 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.850032 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:27.850462 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.851376 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.852565 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.853201 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.853353 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:27.854100 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:27.854104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:27.854104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:27.854443 (+ 339us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:27.854491 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:27.854509 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:27.854584 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:27.854699 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 05:25:27.854783 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260812 05:25:27.855486 30562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48ff29444d0e4c838ca2c55ca2220838. Latest consensus state: current_term: 1 leader_uuid: "48ff29444d0e4c838ca2c55ca2220838" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff29444d0e4c838ca2c55ca2220838" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33987 } } }
I20260812 05:25:27.855573 30562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:27.855747 30564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48ff29444d0e4c838ca2c55ca2220838" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48ff29444d0e4c838ca2c55ca2220838" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33987 } } }
I20260812 05:25:27.855829 30564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:27.856194 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 05:25:27.856436 30577 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:25:27.856493 30577 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:25:27.856559 30578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:27.856750 30578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:27.858724 30578 catalog_manager.cc:1347] Generated new cluster ID: b6adcced599041379521bf84fe77472c
I20260812 05:25:27.858811 30578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:27.876569 30578 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:27.877148 30578 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:27.881067 30578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48ff29444d0e4c838ca2c55ca2220838: Generated new TSK 0
I20260812 05:25:27.881278 30578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:25:27.891651 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:27.887838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57854 (local address 127.24.66.126:33987)
0812 05:25:27.887886 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:27.887888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:27.887984 (+ 96us) server_negotiation.cc:407] Connection header received
0812 05:25:27.888076 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:27.888077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:27.888087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:27.888148 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0812 05:25:27.888303 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.888834 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.889991 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.890196 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.890254 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:27.890486 (+ 232us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 05:25:27.891436 (+ 950us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38977/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 05:25:27.891572 (+ 136us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57854: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38977/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 05:25:27.891724 30560 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57854: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38977/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 05:25:27.891770 30587 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:27.887728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:33987 (local address 127.0.0.1:57854)
0812 05:25:27.887920 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:27.887933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:27.888004 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:27.888171 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:27.888172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:27.888174 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 05:25:27.888235 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:27.888237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.888925 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.888926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:27.889890 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:27.889894 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:27.890289 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:27.890290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:27.890377 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:27.890384 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 05:25:27.891547 (+ 1163us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38977/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 05:25:27.891661 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.66.126:33987: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38977/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":115,"threads_started":1}
I20260812 05:25:27.891950 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 30486
E20260812 05:25:27.905371 30588 webserver.cc:533] Webserver: sslize failed:
E20260812 05:25:27.905480 30588 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T05:25:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (223 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T05:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:27Z Disabled control of system clock
I20260812 05:25:27.945999 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39565
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:46791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:39565
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/testchainca.cert with env {}
W20260812 05:25:28.033558 30593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:28.033811 30593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:28.033838 30593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:28.035238 30593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:28.035290 30593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:28.035305 30593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:28.035320 30593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:28.035334 30593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 05:25:28.035348 30593 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 05:25:28.035410 30593 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 05:25:28.037177 30593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:39565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:28.037473 30593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:28.037724 30593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:28.043426 30601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:28.043447 30598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:28.043426 30599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:28.043869 30593 server_base.cc:1034] running on GCE node
I20260812 05:25:28.044135 30593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:28.044432 30593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:28.045621 30593 hybrid_clock.cc:648] HybridClock initialized: now 1786512328045607 us; error 37 us; skew 500 ppm
I20260812 05:25:28.047187 30593 webserver.cc:460] Webserver started at http://127.24.66.126:45917/ using document root <none> and password file <none>
I20260812 05:25:28.047467 30593 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:28.047549 30593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:28.047711 30593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:28.048822 30593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "3b42c8340d6645f8985440f0497fb7e1"
format_stamp: "Formatted at 2026-08-12 05:25:28 on dist-test-slave-zpfg"
server_key: "f8bfac5d913d24c950130135a86ab741"
server_key_iv: "106e8d9552e56f1648ac3dbfd335c4df"
server_key_version: "encryptionkey@0"
I20260812 05:25:28.049203 30593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "3b42c8340d6645f8985440f0497fb7e1"
format_stamp: "Formatted at 2026-08-12 05:25:28 on dist-test-slave-zpfg"
server_key: "f8bfac5d913d24c950130135a86ab741"
server_key_iv: "106e8d9552e56f1648ac3dbfd335c4df"
server_key_version: "encryptionkey@0"
I20260812 05:25:28.050710 30593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:25:28.051558 30607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:28.051786 30593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:28.051901 30593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "3b42c8340d6645f8985440f0497fb7e1"
format_stamp: "Formatted at 2026-08-12 05:25:28 on dist-test-slave-zpfg"
server_key: "f8bfac5d913d24c950130135a86ab741"
server_key_iv: "106e8d9552e56f1648ac3dbfd335c4df"
server_key_version: "encryptionkey@0"
I20260812 05:25:28.051980 30593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:28.074913 30593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:28.075326 30593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:28.075505 30593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:28.079879 30593 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:39565
I20260812 05:25:28.079944 30660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:39565 every 8 connection(s)
I20260812 05:25:28.080338 30593 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:28.081581 30661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:28.081637 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 30593
I20260812 05:25:28.081741 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:28.081975 24841 external_mini_cluster.cc:1428] Setting key d2958677bb170ee37a392b1f82409d6b
I20260812 05:25:28.085053 30661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1: Bootstrap starting.
I20260812 05:25:28.085738 30661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:28.086104 30661 log.cc:824] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:28.086895 30661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1: No bootstrap required, opened a new log
I20260812 05:25:28.087675 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:28.082594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55502 (local address 127.24.66.126:39565)
0812 05:25:28.082845 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:28.082853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:28.082879 (+ 26us) server_negotiation.cc:407] Connection header received
0812 05:25:28.083356 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:28.083364 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:28.083428 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:28.083586 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:28.083820 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:28.084699 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:28.085582 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:28.086093 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:28.086251 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:28.087022 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:28.087026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:28.087027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:28.087206 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:28.087240 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:28.087251 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:28.087331 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:28.087409 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 05:25:28.087492 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":107,"threads_started":1}
I20260812 05:25:28.088843 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:25:28.089551 30661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b42c8340d6645f8985440f0497fb7e1" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39565 } }
I20260812 05:25:28.089756 30661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:28.089810 30661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b42c8340d6645f8985440f0497fb7e1, State: Initialized, Role: FOLLOWER
I20260812 05:25:28.089956 30661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b42c8340d6645f8985440f0497fb7e1" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39565 } }
I20260812 05:25:28.090039 30661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:28.090065 30661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:28.090139 30661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:28.090827 30661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b42c8340d6645f8985440f0497fb7e1" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39565 } }
I20260812 05:25:28.090940 30661 leader_election.cc:302] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b42c8340d6645f8985440f0497fb7e1; no voters:
I20260812 05:25:28.091116 30661 leader_election.cc:288] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:28.091161 30666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:28.091277 30666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3b42c8340d6645f8985440f0497fb7e1, State: Running, Role: LEADER
I20260812 05:25:28.091369 30666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b42c8340d6645f8985440f0497fb7e1" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39565 } }
I20260812 05:25:28.091403 30661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:28.092592 30667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b42c8340d6645f8985440f0497fb7e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b42c8340d6645f8985440f0497fb7e1" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39565 } } }
I20260812 05:25:28.092804 30667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:28.093158 30668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b42c8340d6645f8985440f0497fb7e1. Latest consensus state: current_term: 1 leader_uuid: "3b42c8340d6645f8985440f0497fb7e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b42c8340d6645f8985440f0497fb7e1" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39565 } } }
I20260812 05:25:28.093302 30668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1 [sys.catalog]: This master's current role is: LEADER
W20260812 05:25:28.093501 30683 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:25:28.093544 30683 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:25:28.093617 30682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:28.093886 30682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:28.095515 30682 catalog_manager.cc:1347] Generated new cluster ID: 23e00c5b8dc5475587c6a6601390145e
I20260812 05:25:28.095572 30682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:28.127902 30682 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:28.128616 30682 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:28.133764 30682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b42c8340d6645f8985440f0497fb7e1: Generated new TSK 0
I20260812 05:25:28.134008 30682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:25:28.636471 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:28.147299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55514 (local address 127.24.66.126:39565)
0812 05:25:28.147349 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:28.147351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:28.147482 (+ 131us) server_negotiation.cc:407] Connection header received
0812 05:25:28.147530 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:28.147531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:28.147544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:28.147644 (+ 100us) server_negotiation.cc:226] Negotiated authn=JWT
0812 05:25:28.147764 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:28.148522 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:28.149337 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:28.149652 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:28.149759 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:28.150167 (+ 408us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 05:25:28.636140 (+485973us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 05:25:28.636404 (+ 264us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55514: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 05:25:28.636545 30664 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55514: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 05:25:28.636620 30691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:28.147250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39565 (local address 127.0.0.1:55514)
0812 05:25:28.147391 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:28.147405 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:28.147477 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:28.147644 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:28.147645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:28.147646 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0812 05:25:28.147700 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:28.147702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:28.148610 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:28.148610 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:28.149233 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:28.149235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:28.149985 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:28.149986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:28.150040 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:28.150045 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 05:25:28.636330 (+486285us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 05:25:28.636482 (+ 152us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.66.126:39565: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260812 05:25:28.637100 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 30593
2026-08-12T05:25:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (720 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (94 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (98 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:28 dist-test-slave-zpfg krb5kdc[30705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:28 dist-test-slave-zpfg krb5kdc[30705](info): set up 2 sockets
Aug 12 05:25:28 dist-test-slave-zpfg krb5kdc[30705](info): commencing operation
krb5kdc: starting...
W20260812 05:25:29.981357 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:25:30.117753 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:37251
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:37251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:30.202826 30718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:30.203064 30718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:30.203087 30718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:30.204218 30718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:30.204255 30718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:30.204283 30718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:30.204298 30718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:30.204309 30718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:30.205770 30718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:37251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:37251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:30.205983 30718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:30.206183 30718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:30.211246 30724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:30.211422 30718 server_base.cc:1034] running on GCE node
W20260812 05:25:30.211251 30727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:30.211388 30725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:30.211807 30718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:30.212174 30718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:30.213369 30718 hybrid_clock.cc:648] HybridClock initialized: now 1786512330213348 us; error 41 us; skew 500 ppm
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:30.216312 30718 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:30.216835 30718 webserver.cc:460] Webserver started at http://127.24.66.126:37923/ using document root <none> and password file <none>
I20260812 05:25:30.217105 30718 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:30.217157 30718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:30.217298 30718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:30.218562 30718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "7228f36724c644f090d1e3c5019b9e08"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "73cdb7926198e7e84c1eff03482586db"
server_key_iv: "5fc20eef3146e56e13df28a838a91182"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.218954 30718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "7228f36724c644f090d1e3c5019b9e08"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "73cdb7926198e7e84c1eff03482586db"
server_key_iv: "5fc20eef3146e56e13df28a838a91182"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.220489 30718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:30.221371 30734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:30.221596 30718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:30.221753 30718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "7228f36724c644f090d1e3c5019b9e08"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "73cdb7926198e7e84c1eff03482586db"
server_key_iv: "5fc20eef3146e56e13df28a838a91182"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.221837 30718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:30.232355 30718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:30.234711 30718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:30.234891 30718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:30.239426 30718 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:37251
I20260812 05:25:30.239464 30789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:37251 every 8 connection(s)
I20260812 05:25:30.239837 30718 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:30.240760 30790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:30.243295 30790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08: Bootstrap starting.
I20260812 05:25:30.243628 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 30718
I20260812 05:25:30.243755 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:30.243947 24841 external_mini_cluster.cc:1428] Setting key 59e79db84bb2cdc26634d529620facf1
I20260812 05:25:30.243947 30790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:30.244277 30790 log.cc:824] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:30.245206 30790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08: No bootstrap required, opened a new log
I20260812 05:25:30.247243 30790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7228f36724c644f090d1e3c5019b9e08" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37251 } }
I20260812 05:25:30.247411 30790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:30.247462 30790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7228f36724c644f090d1e3c5019b9e08, State: Initialized, Role: FOLLOWER
I20260812 05:25:30.247637 30790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7228f36724c644f090d1e3c5019b9e08" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37251 } }
I20260812 05:25:30.247735 30790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:30.247786 30790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:30.247836 30790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:30.248433 30790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7228f36724c644f090d1e3c5019b9e08" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37251 } }
I20260812 05:25:30.248575 30790 leader_election.cc:302] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7228f36724c644f090d1e3c5019b9e08; no voters:
I20260812 05:25:30.248807 30790 leader_election.cc:288] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:30.248835 30795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:30.249043 30795 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [term 1 LEADER]: Becoming Leader. State: Replica: 7228f36724c644f090d1e3c5019b9e08, State: Running, Role: LEADER
I20260812 05:25:30.249202 30795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7228f36724c644f090d1e3c5019b9e08" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37251 } }
I20260812 05:25:30.249537 30790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:30.249577 30797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7228f36724c644f090d1e3c5019b9e08. Latest consensus state: current_term: 1 leader_uuid: "7228f36724c644f090d1e3c5019b9e08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7228f36724c644f090d1e3c5019b9e08" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37251 } } }
I20260812 05:25:30.249650 30797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:30.249877 30796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7228f36724c644f090d1e3c5019b9e08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7228f36724c644f090d1e3c5019b9e08" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37251 } } }
I20260812 05:25:30.249982 30796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:30.250221 30799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:30.250814 30799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:30.252815 30799 catalog_manager.cc:1347] Generated new cluster ID: a8294984a06144569d733aac9b76b558
I20260812 05:25:30.252873 30799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:30.255528 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:30.244606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.24.66.126:37251)
0812 05:25:30.244769 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:30.244780 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:30.244813 (+ 33us) server_negotiation.cc:407] Connection header received
0812 05:25:30.245287 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:30.245296 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:30.245364 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:30.245505 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:30.246231 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.247222 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.248203 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.248721 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.248825 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:30.251913 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:30.251916 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:30.251917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:30.251929 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:30.253906 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.254310 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.254313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.254314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.254447 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.254663 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.254664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.254664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.254918 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:30.255121 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:30.255201 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 05:25:30.255342 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260812 05:25:30.296415 30799 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:30.297273 30799 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:30.307153 30799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7228f36724c644f090d1e3c5019b9e08: Generated new TSK 0
I20260812 05:25:30.307457 30799 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:25:30.346823 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:37251
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:30.450865 30816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:30.451077 30816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:30.451102 30816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:30.452242 30816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:30.452281 30816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:30.452307 30816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:30.453855 30816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:30.454053 30816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:30.454257 30816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:30.459537 30825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:30.459563 30823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:30.459573 30822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:30.459857 30816 server_base.cc:1034] running on GCE node
I20260812 05:25:30.460078 30816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:30.460295 30816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:30.461536 30816 hybrid_clock.cc:648] HybridClock initialized: now 1786512330461512 us; error 31 us; skew 500 ppm
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:30.464545 30816 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:25:30.465123 30816 webserver.cc:460] Webserver started at http://127.24.66.65:34625/ using document root <none> and password file <none>
I20260812 05:25:30.465449 30816 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:30.465533 30816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:30.465691 30816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:30.466892 30816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "bc20462043834303b1304ffda06dd90d"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "3c0a4a35b88b62aba14545c8d26d783f"
server_key_iv: "ff0b626d6bd73fe1b3f47af1f0486b76"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.467271 30816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "bc20462043834303b1304ffda06dd90d"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "3c0a4a35b88b62aba14545c8d26d783f"
server_key_iv: "ff0b626d6bd73fe1b3f47af1f0486b76"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.468770 30816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:30.469671 30832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:30.469866 30816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:30.469980 30816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "bc20462043834303b1304ffda06dd90d"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "3c0a4a35b88b62aba14545c8d26d783f"
server_key_iv: "ff0b626d6bd73fe1b3f47af1f0486b76"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.470058 30816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:30.488562 30816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:30.491251 30816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:30.491496 30816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:30.491765 30816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:30.492103 30816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:30.492161 30816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:30.492208 30816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:30.492254 30816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:30.499029 30816 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:37375
I20260812 05:25:30.499099 30959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:37375 every 8 connection(s)
I20260812 05:25:30.499459 30816 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:30.504550 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 30816
I20260812 05:25:30.504691 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:30.504911 24841 external_mini_cluster.cc:1428] Setting key 1620601f92a148818b6f6fe2f8475215
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:30.511317 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:30.500511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:34437 (local address 127.24.66.126:37251)
0812 05:25:30.500611 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:30.500614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:30.500779 (+ 165us) server_negotiation.cc:407] Connection header received
0812 05:25:30.501891 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:30.501893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:30.501905 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:30.501964 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:30.502700 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.503238 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.504292 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.504537 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.504625 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:30.507263 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:30.507265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:30.507265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:30.507273 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:30.509279 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.509862 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.509863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.509864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.509926 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.510294 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.510295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.510296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.510487 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:30.510547 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:30.510975 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 05:25:30.511121 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 05:25:30.511464 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:30.500368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:37251 (local address 127.24.66.65:34437)
0812 05:25:30.500624 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:30.500683 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:30.501761 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:30.502054 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:30.502058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:30.502368 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:30.502572 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:30.502578 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.503343 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.503344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:30.504193 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:30.504195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.504638 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.504639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:30.504823 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:30.505478 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:30.505489 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:30.507149 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:30.509384 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:30.509394 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:30.509397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:30.509761 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:30.510005 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:30.510006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:30.510006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:30.510187 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:30.510557 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:30.510567 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:30.510779 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 05:25:30.510920 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 05:25:30.511132 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":138,"threads_started":1}
I20260812 05:25:30.512123 30960 heartbeater.cc:347] Connected to a master server at 127.24.66.126:37251
I20260812 05:25:30.512257 30960 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:30.512493 30960 heartbeater.cc:511] Master 127.24.66.126:37251 requested a full tablet report, sending...
I20260812 05:25:30.513115 30751 ts_manager.cc:194] Registered new tserver with Master: bc20462043834303b1304ffda06dd90d (127.24.66.65:37375)
I20260812 05:25:30.513785 30751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:34437
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:25:30.539253 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:37251
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:30.628582 30965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:30.628808 30965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:30.628834 30965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:30.630095 30965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:30.630146 30965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:30.630177 30965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:30.631868 30965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:30.632146 30965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:30.632403 30965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:30.638321 30971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:30.638324 30970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:30.638324 30973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:30.638370 30965 server_base.cc:1034] running on GCE node
I20260812 05:25:30.638854 30965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:30.639124 30965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:30.640357 30965 hybrid_clock.cc:648] HybridClock initialized: now 1786512330640343 us; error 30 us; skew 500 ppm
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:30.643260 30965 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:25:30.643765 30965 webserver.cc:460] Webserver started at http://127.24.66.66:37337/ using document root <none> and password file <none>
I20260812 05:25:30.644011 30965 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:30.644083 30965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:30.644251 30965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:30.645568 30965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "9d85d7858e594d5ba84bd404b8ccf14e"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "b11aaa40f7ead99014b6f9dbe639059c"
server_key_iv: "325208b534835ac118b0584e54c5cc39"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.645962 30965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "9d85d7858e594d5ba84bd404b8ccf14e"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "b11aaa40f7ead99014b6f9dbe639059c"
server_key_iv: "325208b534835ac118b0584e54c5cc39"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.647552 30965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:25:30.648447 30980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:30.648636 30965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:30.648787 30965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "9d85d7858e594d5ba84bd404b8ccf14e"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "b11aaa40f7ead99014b6f9dbe639059c"
server_key_iv: "325208b534835ac118b0584e54c5cc39"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.648869 30965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:30.666136 30965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:30.668717 30965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:30.668915 30965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:30.669198 30965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:30.669531 30965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:30.669587 30965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:30.669636 30965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:30.669675 30965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:30.676721 30965 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:40163
I20260812 05:25:30.676788 31103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:40163 every 8 connection(s)
I20260812 05:25:30.677192 30965 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:30.686305 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 30965
I20260812 05:25:30.686450 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:30.686676 24841 external_mini_cluster.cc:1428] Setting key 9b30806addc0f3ba3e9cd3f1cc132fb6
I20260812 05:25:30.688004 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:30.678046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:46061 (local address 127.24.66.126:37251)
0812 05:25:30.678153 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:30.678156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:30.678302 (+ 146us) server_negotiation.cc:407] Connection header received
0812 05:25:30.679326 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:30.679327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:30.679340 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:30.679439 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:30.680131 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.680688 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.681724 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.681925 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.682011 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:30.684533 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:30.684535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:30.684535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:30.684542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:30.685977 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.686604 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.686605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.686605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.686687 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.687132 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.687133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.687134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.687300 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:30.687378 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:30.687781 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 05:25:30.687880 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 05:25:30.688184 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:30.677984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:37251 (local address 127.24.66.66:46061)
0812 05:25:30.678170 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:30.678215 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:30.679196 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:30.679549 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:30.679553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:30.679826 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:30.680025 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:30.680029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.680792 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.680793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:30.681621 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:30.681622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.682021 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.682022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:30.682173 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:30.682766 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:30.682775 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:30.684409 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:30.686085 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:30.686094 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:30.686097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:30.686474 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:30.686828 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:30.686829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:30.686830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:30.686988 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:30.687377 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:30.687382 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:30.687567 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 05:25:30.687684 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 05:25:30.687896 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260812 05:25:30.688645 31104 heartbeater.cc:347] Connected to a master server at 127.24.66.126:37251
I20260812 05:25:30.688746 31104 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:30.688964 31104 heartbeater.cc:511] Master 127.24.66.126:37251 requested a full tablet report, sending...
I20260812 05:25:30.689375 30751 ts_manager.cc:194] Registered new tserver with Master: 9d85d7858e594d5ba84bd404b8ccf14e (127.24.66.66:40163)
I20260812 05:25:30.689980 30751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:46061
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:25:30.718067 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:37251
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:30.809013 31109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:30.809234 31109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:30.809259 31109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:30.810520 31109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:30.810570 31109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:30.810598 31109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:30.812155 31109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:30.812422 31109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:30.812680 31109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:30.818212 31114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:30.818221 31118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:30.818480 31115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:30.818631 31109 server_base.cc:1034] running on GCE node
I20260812 05:25:30.818871 31109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:30.819170 31109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:30.820358 31109 hybrid_clock.cc:648] HybridClock initialized: now 1786512330820347 us; error 34 us; skew 500 ppm
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:30.823418 31109 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:25:30.824018 31109 webserver.cc:460] Webserver started at http://127.24.66.67:38397/ using document root <none> and password file <none>
I20260812 05:25:30.824366 31109 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:30.824450 31109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:30.824611 31109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:30.825771 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "86720f3dbe904f42a2a1329205433871"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "ab33ffd02d46453494f0133c3e6a429b"
server_key_iv: "035b35f3d6d87e53fc7206fc39f48cce"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.826165 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "86720f3dbe904f42a2a1329205433871"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "ab33ffd02d46453494f0133c3e6a429b"
server_key_iv: "035b35f3d6d87e53fc7206fc39f48cce"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.827652 31109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:30.828715 31125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:30.829001 31109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:30.829139 31109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "86720f3dbe904f42a2a1329205433871"
format_stamp: "Formatted at 2026-08-12 05:25:30 on dist-test-slave-zpfg"
server_key: "ab33ffd02d46453494f0133c3e6a429b"
server_key_iv: "035b35f3d6d87e53fc7206fc39f48cce"
server_key_version: "encryptionkey@0"
I20260812 05:25:30.829221 31109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:30.849077 31109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:30.852031 31109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:30.852249 31109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:30.852552 31109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:30.853055 31109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:30.853106 31109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:30.853193 31109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:30.853253 31109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:30.860126 31109 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:39699
I20260812 05:25:30.860168 31246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:39699 every 8 connection(s)
I20260812 05:25:30.860683 31109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:30.864480 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 31109
I20260812 05:25:30.864619 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:30.864801 24841 external_mini_cluster.cc:1428] Setting key 8119d5fa076c6f1ebeda3916144068b1
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:30.871279 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:30.861626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:51671 (local address 127.24.66.126:37251)
0812 05:25:30.861709 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:30.861711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:30.861880 (+ 169us) server_negotiation.cc:407] Connection header received
0812 05:25:30.862954 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:30.862956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:30.862969 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:30.863037 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:30.863755 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.864443 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.865944 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.866164 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.866225 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:30.868571 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:30.868572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:30.868572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:30.868580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:30.869956 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.870406 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.870407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.870407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.870471 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.870709 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.870709 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.870709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.870834 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:30.870889 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:30.871133 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 05:25:30.871202 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:25:30.871460 31249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:30.861543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:37251 (local address 127.24.66.67:51671)
0812 05:25:30.861727 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:30.861771 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:30.862812 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:30.863116 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:30.863120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:30.863429 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:30.863636 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:30.863639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.864561 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.864562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:30.865870 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:30.865873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.866248 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.866249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:30.866431 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:30.866959 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:30.866968 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:30.868470 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:30.870041 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:30.870048 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:30.870050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:30.870277 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:30.870542 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:30.870543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:30.870543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:30.870649 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:30.870892 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:30.870897 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:30.871017 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 05:25:30.871101 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 05:25:30.871235 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
I20260812 05:25:30.871953 31247 heartbeater.cc:347] Connected to a master server at 127.24.66.126:37251
I20260812 05:25:30.872042 31247 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:30.872198 31247 heartbeater.cc:511] Master 127.24.66.126:37251 requested a full tablet report, sending...
I20260812 05:25:30.872608 30751 ts_manager.cc:194] Registered new tserver with Master: 86720f3dbe904f42a2a1329205433871 (127.24.66.67:39699)
I20260812 05:25:30.873018 30751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:51671
I20260812 05:25:30.876613 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/test-admin.keytab.
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:30.896530 24841 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 05:25:30.898773 31254 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:30.897626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39230 (local address 127.24.66.65:37375)
0812 05:25:30.897797 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:30.897805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:30.897821 (+ 16us) server_negotiation.cc:407] Connection header received
0812 05:25:30.897850 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:30.897852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:30.897866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:30.897938 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:30.898669 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39230: BlockingRecv error: recv got EOF from 127.0.0.1:39230 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
W20260812 05:25:30.898838 24841 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/krb5cc' of type FILE
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:30.899973 24841 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:30.900962 24841 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:25:30.909039 31254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:30.901505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39232 (local address 127.24.66.65:37375)
0812 05:25:30.901569 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:30.901570 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:30.901578 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:30.901636 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:30.901637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:30.901648 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:30.901703 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:30.902125 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.902958 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.903682 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.904055 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.904115 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:30.906192 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:30.906202 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:30.906202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:30.906209 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:30.907937 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.908297 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.908300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.908301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.908370 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.908561 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.908562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.908562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.908709 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:30.908837 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:30.908894 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 05:25:30.908963 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 05:25:30.911255 31254 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:30.910647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39242 (local address 127.24.66.65:37375)
0812 05:25:30.910713 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:30.910716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:30.910751 (+ 35us) server_negotiation.cc:407] Connection header received
0812 05:25:30.910827 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:30.910828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:30.910842 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:30.910917 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:30.911211 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39242: BlockingRecv error: recv got EOF from 127.0.0.1:39242 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 05:25:30.911440 24841 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/krb5cc' of type FILE
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:30.912551 24841 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:30.913733 24841 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:25:30.921561 31254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:30.914428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39244 (local address 127.24.66.65:37375)
0812 05:25:30.914476 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:30.914478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:30.914541 (+ 63us) server_negotiation.cc:407] Connection header received
0812 05:25:30.914618 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:30.914619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:30.914632 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:30.914698 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:30.915099 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.915635 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.916358 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.916564 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.916644 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:30.918998 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:30.919000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:30.919000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:30.919008 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:30.920309 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.920726 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.920727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.920728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.920784 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.920981 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.920981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.920982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.921078 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:30.921149 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:30.921333 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 05:25:30.921402 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 05:25:30.923630 31254 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:30.922899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39248 (local address 127.24.66.65:37375)
0812 05:25:30.922968 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:30.922970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:30.922998 (+ 28us) server_negotiation.cc:407] Connection header received
0812 05:25:30.923072 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:30.923073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:30.923082 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:30.923131 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:30.923576 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39248: BlockingRecv error: recv got EOF from 127.0.0.1:39248 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 05:25:30.923873 24841 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786512301961891-24841-0/krb5kdc/krb5cc' of type FILE
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:30.925242 24841 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:30.926311 24841 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 05:25:30 dist-test-slave-zpfg krb5kdc[30705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:25:30.933718 31254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:30.927058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39252 (local address 127.24.66.65:37375)
0812 05:25:30.927195 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:30.927197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:30.927205 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:30.927226 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:30.927226 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:30.927235 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:30.927285 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:30.927669 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.928261 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.928994 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:30.929179 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:30.929240 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:30.931390 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:30.931391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:30.931392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:30.931398 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:30.932708 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.933051 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.933051 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.933052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.933105 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:30.933309 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:30.933310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:30.933310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:30.933423 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:30.933515 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:30.933593 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 05:25:30.933652 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:25:30.934486 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 30816
I20260812 05:25:30.941555 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 30965
I20260812 05:25:30.948534 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 31109
I20260812 05:25:30.954469 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 30718
2026-08-12T05:25:30Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2125 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:31 dist-test-slave-zpfg krb5kdc[31267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:31 dist-test-slave-zpfg krb5kdc[31267](info): set up 2 sockets
Aug 12 05:25:31 dist-test-slave-zpfg krb5kdc[31267](info): commencing operation
krb5kdc: starting...
W20260812 05:25:32.039935 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:32 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:32Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.126.keytab.
Entry for principal oryx/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.126.keytab.
I20260812 05:25:32.177979 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:45023
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.126.keytab
--principal=oryx/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:45023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:32.265671 31280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:32.265981 31280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:32.266031 31280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:32.267330 31280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:32.267412 31280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:32.267436 31280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:32.267457 31280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:32.267489 31280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:32.269098 31280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:45023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.126.keytab
--principal=oryx/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:45023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:32.269402 31280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:32.269657 31280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:32.275010 31285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:32.275046 31289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:32.275030 31286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:32.275344 31280 server_base.cc:1034] running on GCE node
I20260812 05:25:32.275709 31280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:32.275995 31280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:32.277230 31280 hybrid_clock.cc:648] HybridClock initialized: now 1786512332277204 us; error 45 us; skew 500 ppm
Aug 12 05:25:32 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:32.279927 31280 init.cc:377] Logged in from keytab as oryx/127.24.66.126@KRBTEST.COM (short username oryx)
I20260812 05:25:32.280468 31280 webserver.cc:460] Webserver started at http://127.24.66.126:45703/ using document root <none> and password file <none>
I20260812 05:25:32.280668 31280 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:32.280709 31280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:32.280807 31280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:32.281896 31280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "0ec15ce8ef3740b6abd2322926963037"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "7b5c217f3b7e4b11c835ffc6f31c2d21"
server_key_iv: "852ad50e9db6cc82c67d62c807b3f65e"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.282220 31280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "0ec15ce8ef3740b6abd2322926963037"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "7b5c217f3b7e4b11c835ffc6f31c2d21"
server_key_iv: "852ad50e9db6cc82c67d62c807b3f65e"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.283663 31280 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:32.284725 31297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:32.284927 31280 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:32.285022 31280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "0ec15ce8ef3740b6abd2322926963037"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "7b5c217f3b7e4b11c835ffc6f31c2d21"
server_key_iv: "852ad50e9db6cc82c67d62c807b3f65e"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.285085 31280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:32.319890 31280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:32.322710 31280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:32.322908 31280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:32.327250 31280 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:45023
I20260812 05:25:32.327313 31352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:45023 every 8 connection(s)
I20260812 05:25:32.327685 31280 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:32.328864 31354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:32.331562 31354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037: Bootstrap starting.
I20260812 05:25:32.332266 31354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:32.332620 31354 log.cc:824] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:32.333469 31354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037: No bootstrap required, opened a new log
I20260812 05:25:32.334884 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 31280
I20260812 05:25:32.334971 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:32.335181 24841 external_mini_cluster.cc:1428] Setting key 51760b551154613be21fd5ecd936070b
I20260812 05:25:32.335727 31354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } }
I20260812 05:25:32.335952 31354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:32.336030 31354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec15ce8ef3740b6abd2322926963037, State: Initialized, Role: FOLLOWER
I20260812 05:25:32.336220 31354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } }
I20260812 05:25:32.336311 31354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:32.336357 31354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:32.336414 31354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:32.337741 31354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } }
I20260812 05:25:32.337848 31354 leader_election.cc:302] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ec15ce8ef3740b6abd2322926963037; no voters:
I20260812 05:25:32.338035 31354 leader_election.cc:288] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:32.338073 31359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:32.338228 31359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 1 LEADER]: Becoming Leader. State: Replica: 0ec15ce8ef3740b6abd2322926963037, State: Running, Role: LEADER
I20260812 05:25:32.338387 31354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:32.338402 31359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } }
I20260812 05:25:32.338752 31359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ec15ce8ef3740b6abd2322926963037. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } }
I20260812 05:25:32.338923 31359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:32.338771 31361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } }
I20260812 05:25:32.339205 31361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:32.339720 31367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:32.340297 31367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:32.342092 31367 catalog_manager.cc:1347] Generated new cluster ID: d3b38d4bde584a1d9c334521c2e37ce6
I20260812 05:25:32.342147 31367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:25:32 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:32.347510 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:32.335800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.24.66.126:45023)
0812 05:25:32.336070 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:32.336119 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:32.336154 (+ 35us) server_negotiation.cc:407] Connection header received
0812 05:25:32.336295 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:32.336300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:32.336423 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:32.336564 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:32.337483 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.338772 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.340108 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.340647 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.340786 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:32.343742 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:32.343747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:32.343747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:32.343761 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:32.345688 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:32.346122 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:32.346125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:32.346126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:32.346257 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:32.346523 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:32.346525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:32.346525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:32.346803 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:32.346993 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:32.347116 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 05:25:32.347279 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":111,"threads_started":1}
I20260812 05:25:32.363683 31367 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:32.364271 31367 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:32.383339 31367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037: Generated new TSK 0
I20260812 05:25:32.383597 31367 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab.
Entry for principal oryx/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab.
I20260812 05:25:32.422777 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45023
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab
--principal=oryx/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:32.526525 31381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:32.526775 31381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:32.526819 31381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:32.528052 31381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:32.528121 31381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:32.528169 31381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:32.529798 31381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab
--principal=oryx/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:32.530081 31381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:32.530371 31381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:32.535864 31387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:32.535988 31381 server_base.cc:1034] running on GCE node
W20260812 05:25:32.535871 31391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:32.536075 31388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:32.536419 31381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:32.536721 31381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:32.537891 31381 hybrid_clock.cc:648] HybridClock initialized: now 1786512332537874 us; error 42 us; skew 500 ppm
Aug 12 05:25:32 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:32.540596 31381 init.cc:377] Logged in from keytab as oryx/127.24.66.65@KRBTEST.COM (short username oryx)
I20260812 05:25:32.541040 31381 webserver.cc:460] Webserver started at http://127.24.66.65:33059/ using document root <none> and password file <none>
I20260812 05:25:32.541343 31381 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:32.541415 31381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:32.541569 31381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:32.542716 31381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "044a7521d21e4c0b93be61cd66d6bea3"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "caca04f4544b04a444891e79639d08e2"
server_key_iv: "ff6e20eab998e6714ad873efc32eda36"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.543087 31381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "044a7521d21e4c0b93be61cd66d6bea3"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "caca04f4544b04a444891e79639d08e2"
server_key_iv: "ff6e20eab998e6714ad873efc32eda36"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.544463 31381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:32.545228 31399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:32.545436 31381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:32.545540 31381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "044a7521d21e4c0b93be61cd66d6bea3"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "caca04f4544b04a444891e79639d08e2"
server_key_iv: "ff6e20eab998e6714ad873efc32eda36"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.545617 31381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:32.557155 31381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:32.559492 31381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:32.559669 31381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:32.559923 31381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:32.560269 31381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:32.560334 31381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:32.560372 31381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:32.560416 31381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:32.566735 31381 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:42989
I20260812 05:25:32.566788 31514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:42989 every 8 connection(s)
I20260812 05:25:32.567134 31381 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:32.569686 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 31381
I20260812 05:25:32.569803 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:32.570060 24841 external_mini_cluster.cc:1428] Setting key e0e02ede7e612e8e6ea3345349b722c8
Aug 12 05:25:32 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:32.579785 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:32.567843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:37579 (local address 127.24.66.126:45023)
0812 05:25:32.567911 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:32.567913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:32.568144 (+ 231us) server_negotiation.cc:407] Connection header received
0812 05:25:32.568761 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:32.568762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:32.568782 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:32.568848 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:32.570073 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.570857 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.572456 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.572748 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.572860 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:32.575938 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:32.575940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:32.575941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:32.575952 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:32.578074 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:32.578658 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:32.578660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:32.578660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:32.578750 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:32.579030 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:32.579031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:32.579031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:32.579207 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:32.579285 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:32.579561 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 05:25:32.579661 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:25:32.579877 31518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:32.567825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.65:37579)
0812 05:25:32.567986 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:32.568048 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:32.568632 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:32.568926 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:32.568933 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:32.569631 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:32.569953 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:32.569961 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.570965 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.570966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:32.572338 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:32.572341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.572876 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.572878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:32.573045 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:32.573744 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:32.573756 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:32.575764 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:32.578195 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:32.578204 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:32.578207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:32.578572 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:32.578840 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:32.578841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:32.578841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:32.578957 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:32.579279 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:32.579284 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:32.579417 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 05:25:32.579504 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 05:25:32.579646 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260812 05:25:32.580479 31516 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45023
I20260812 05:25:32.580600 31516 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:32.580830 31516 heartbeater.cc:511] Master 127.24.66.126:45023 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.24.66.66@KRBTEST.COM; defaulting to no policy
I20260812 05:25:32.581506 31315 ts_manager.cc:194] Registered new tserver with Master: 044a7521d21e4c0b93be61cd66d6bea3 (127.24.66.65:42989)
I20260812 05:25:32.582161 31315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:37579
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab.
Entry for principal oryx/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab.
I20260812 05:25:32.601464 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45023
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab
--principal=oryx/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:32.686663 31521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:32.686926 31521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:32.686966 31521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:32.688156 31521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:32.688239 31521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:32.688287 31521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:32.689802 31521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab
--principal=oryx/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:32.690073 31521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:32.690333 31521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:32.695852 31528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:32.695863 31529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:32.695963 31531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:32.696411 31521 server_base.cc:1034] running on GCE node
I20260812 05:25:32.696596 31521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:32.696806 31521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:32.697947 31521 hybrid_clock.cc:648] HybridClock initialized: now 1786512332697926 us; error 26 us; skew 500 ppm
Aug 12 05:25:32 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:32.700680 31521 init.cc:377] Logged in from keytab as oryx/127.24.66.66@KRBTEST.COM (short username oryx)
I20260812 05:25:32.701223 31521 webserver.cc:460] Webserver started at http://127.24.66.66:40657/ using document root <none> and password file <none>
I20260812 05:25:32.701502 31521 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:32.701576 31521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:32.701732 31521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:32.702905 31521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "9e05066fbe2f217e37d4c4a80966555f"
server_key_iv: "c38a07cdacbbe1f4c592d6116763e323"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.703311 31521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "9e05066fbe2f217e37d4c4a80966555f"
server_key_iv: "c38a07cdacbbe1f4c592d6116763e323"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.704732 31521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:32.705775 31538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:32.706051 31521 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:32.706175 31521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "9e05066fbe2f217e37d4c4a80966555f"
server_key_iv: "c38a07cdacbbe1f4c592d6116763e323"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.706265 31521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:32.732297 31521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:32.734803 31521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:32.734984 31521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:32.735267 31521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:32.735612 31521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:32.735671 31521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:32.735714 31521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:32.735742 31521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:32.742249 31521 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:38509
I20260812 05:25:32.742300 31657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:38509 every 8 connection(s)
I20260812 05:25:32.742712 31521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:32.748138 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 31521
I20260812 05:25:32.748276 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:32.748646 24841 external_mini_cluster.cc:1428] Setting key b42f2c4594050b541dfeee82234c7f75
Aug 12 05:25:32 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:32.754631 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:32.743572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:59449 (local address 127.24.66.126:45023)
0812 05:25:32.743713 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:32.743715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:32.743943 (+ 228us) server_negotiation.cc:407] Connection header received
0812 05:25:32.744595 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:32.744596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:32.744611 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:32.744670 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:32.745770 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.746419 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.747588 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.747827 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.747935 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:32.751037 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:32.751039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:32.751039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:32.751047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:32.752993 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:32.753454 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:32.753456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:32.753456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:32.753524 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:32.753837 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:32.753838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:32.753838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:32.753998 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:32.754073 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:32.754402 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 05:25:32.754505 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 05:25:32.754899 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:32.743486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.66:59449)
0812 05:25:32.743798 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:32.743839 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:32.744455 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:32.744771 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:32.744775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:32.745420 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:32.745639 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:32.745644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.746528 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.746529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:32.747483 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:32.747486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.747946 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.747947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:32.748085 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:32.748841 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:32.748850 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:32.750919 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:32.753107 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:32.753115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:32.753117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:32.753366 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:32.753603 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:32.753604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:32.753604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:32.753765 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:32.754090 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:32.754097 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:32.754234 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 05:25:32.754398 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 05:25:32.754605 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":95,"threads_started":1}
I20260812 05:25:32.755471 31658 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45023
I20260812 05:25:32.755599 31658 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:32.755846 31658 heartbeater.cc:511] Master 127.24.66.126:45023 requested a full tablet report, sending...
I20260812 05:25:32.756309 31315 ts_manager.cc:194] Registered new tserver with Master: 7e191f23bc0c4e04aa4ec9cc21fbf4f4 (127.24.66.66:38509)
I20260812 05:25:32.756740 31315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:59449
WARNING: no policy specified for oryx/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab.
Entry for principal oryx/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab.
I20260812 05:25:32.781584 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45023
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab
--principal=oryx/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:32.877413 31663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:32.877733 31663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:32.877767 31663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:32.879148 31663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:32.879194 31663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:32.879221 31663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:32.880630 31663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab
--principal=oryx/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:32.880820 31663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:32.880999 31663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:32.886523 31669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:32.886584 31671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:32.886633 31663 server_base.cc:1034] running on GCE node
W20260812 05:25:32.886822 31668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:32.887053 31663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:32.887349 31663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:32.888543 31663 hybrid_clock.cc:648] HybridClock initialized: now 1786512332888505 us; error 28 us; skew 500 ppm
Aug 12 05:25:32 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:32.891323 31663 init.cc:377] Logged in from keytab as oryx/127.24.66.67@KRBTEST.COM (short username oryx)
I20260812 05:25:32.891780 31663 webserver.cc:460] Webserver started at http://127.24.66.67:45111/ using document root <none> and password file <none>
I20260812 05:25:32.892025 31663 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:32.892097 31663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:32.892254 31663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:32.893430 31663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "86dc52beff2747f9bbe8f6041061d1a1"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "d38675db716a712eb43ffc36fdbaab13"
server_key_iv: "9daff91392a8f47da9905d9df50b8ed2"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.893829 31663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "86dc52beff2747f9bbe8f6041061d1a1"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "d38675db716a712eb43ffc36fdbaab13"
server_key_iv: "9daff91392a8f47da9905d9df50b8ed2"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.895480 31663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:32.896376 31680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:32.896552 31663 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:32.896659 31663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "86dc52beff2747f9bbe8f6041061d1a1"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "d38675db716a712eb43ffc36fdbaab13"
server_key_iv: "9daff91392a8f47da9905d9df50b8ed2"
server_key_version: "encryptionkey@0"
I20260812 05:25:32.896735 31663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:32.926654 31663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:32.929286 31663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:32.929477 31663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:32.929764 31663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:32.930111 31663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:32.930171 31663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:32.930209 31663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:32.930256 31663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:32.936664 31663 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:34223
I20260812 05:25:32.936751 31799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:34223 every 8 connection(s)
I20260812 05:25:32.937105 31663 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:32.938656 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 31663
I20260812 05:25:32.938776 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:32.938969 24841 external_mini_cluster.cc:1428] Setting key f9ac5ff15b405b049e15d61cd7908139
Aug 12 05:25:32 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:32.947154 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:32.937977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:38369 (local address 127.24.66.126:45023)
0812 05:25:32.938038 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:32.938041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:32.938222 (+ 181us) server_negotiation.cc:407] Connection header received
0812 05:25:32.938845 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:32.938847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:32.938860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:32.938934 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:32.940057 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.940619 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.941653 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.941879 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.941963 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:32.944320 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:32.944322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:32.944322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:32.944330 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:32.945609 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:32.946078 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:32.946079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:32.946079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:32.946141 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:32.946447 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:32.946448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:32.946448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:32.946573 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:32.946654 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:32.946952 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 05:25:32.947041 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 05:25:32.947384 31803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:32.937861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.67:38369)
0812 05:25:32.938066 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:32.938122 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:32.938696 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:32.939009 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:32.939061 (+ 52us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:32.939735 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:32.939958 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:32.939963 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.940731 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.940732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:32.941554 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:32.941556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:32.941971 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:32.941972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:32.942108 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:32.942704 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:32.942712 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:32.944179 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:32.945699 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:32.945708 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:32.945710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:32.945974 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:32.946222 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:32.946223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:32.946223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:32.946390 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:32.946643 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:32.946650 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:32.946807 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 05:25:32.946915 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 05:25:32.947093 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":71,"threads_started":1}
I20260812 05:25:32.947904 31801 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45023
I20260812 05:25:32.948040 31801 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:32.948205 31801 heartbeater.cc:511] Master 127.24.66.126:45023 requested a full tablet report, sending...
I20260812 05:25:32.948622 31315 ts_manager.cc:194] Registered new tserver with Master: 86dc52beff2747f9bbe8f6041061d1a1 (127.24.66.67:34223)
I20260812 05:25:32.949115 31315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:38369
I20260812 05:25:32.950709 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.66.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.125.keytab.
Entry for principal oryx/127.24.66.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.125.keytab.
I20260812 05:25:32.982961 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.125:44217
--webserver_interface=127.24.66.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.125.keytab
--principal=oryx/127.24.66.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:45023,127.24.66.125:44217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:33.071440 31806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:33.071666 31806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:33.071691 31806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:33.072793 31806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:33.072830 31806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:33.072844 31806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:33.072857 31806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:33.072870 31806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:33.074564 31806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:45023,127.24.66.125:44217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.125.keytab
--principal=oryx/127.24.66.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.125:44217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.66.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
W20260812 05:25:33.074680 31806 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.66.126:45023,127.24.66.125:44217'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 05:25:33.074797 31806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:33.074968 31806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:33.080518 31813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:33.080539 31815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:33.080526 31812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:33.080961 31806 server_base.cc:1034] running on GCE node
I20260812 05:25:33.081256 31806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:33.081547 31806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:33.082715 31806 hybrid_clock.cc:648] HybridClock initialized: now 1786512333082701 us; error 29 us; skew 500 ppm
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:33.085439 31806 init.cc:377] Logged in from keytab as oryx/127.24.66.125@KRBTEST.COM (short username oryx)
I20260812 05:25:33.085964 31806 webserver.cc:460] Webserver started at http://127.24.66.125:44787/ using document root <none> and password file <none>
I20260812 05:25:33.086199 31806 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:33.086269 31806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:33.086477 31806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:33.087612 31806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data/instance:
uuid: "63fb1e36d61646058cb8282b12a09e8e"
format_stamp: "Formatted at 2026-08-12 05:25:33 on dist-test-slave-zpfg"
server_key: "db3337215908f1013190bc78e362a454"
server_key_iv: "60eb96a6be9b4c15dacdedaa7487eda1"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.087990 31806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/wal/instance:
uuid: "63fb1e36d61646058cb8282b12a09e8e"
format_stamp: "Formatted at 2026-08-12 05:25:33 on dist-test-slave-zpfg"
server_key: "db3337215908f1013190bc78e362a454"
server_key_iv: "60eb96a6be9b4c15dacdedaa7487eda1"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.089367 31806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:25:33.090140 31823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.090306 31806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:25:33.090435 31806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/wal
uuid: "63fb1e36d61646058cb8282b12a09e8e"
format_stamp: "Formatted at 2026-08-12 05:25:33 on dist-test-slave-zpfg"
server_key: "db3337215908f1013190bc78e362a454"
server_key_iv: "60eb96a6be9b4c15dacdedaa7487eda1"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.090538 31806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:33.110450 31806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:33.112550 31806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:33.112730 31806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:33.117424 31806 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.125:44217
I20260812 05:25:33.117463 31879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.125:44217 every 8 connection(s)
I20260812 05:25:33.117844 31806 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
I20260812 05:25:33.119006 31880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:33.119247 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 31806
I20260812 05:25:33.121578 31880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.66.125@KRBTEST.COM
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.125@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:33.130300 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.122382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.24.66.126:45023)
0812 05:25:33.122441 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.122444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.122639 (+ 195us) server_negotiation.cc:407] Connection header received
0812 05:25:33.122695 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.122696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.122708 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.122760 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.123232 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.123729 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.124536 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.124743 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.124836 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.127493 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.127495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.127495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.127503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.128872 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.129268 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.129269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.129269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.129332 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.129581 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.129582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.129582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.129751 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.129873 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.130085 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.130178 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:25:33.130498 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.120086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45692 (local address 127.24.66.125:44217)
0812 05:25:33.120294 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.120305 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.120340 (+ 35us) server_negotiation.cc:407] Connection header received
0812 05:25:33.120815 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.120824 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.120899 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.121061 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.121706 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.122935 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.123708 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.124121 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.124242 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.127059 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.127067 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.127067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.127075 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.128882 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.129281 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.129284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.129284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.129377 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.129605 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.129606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.129607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.129834 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.130021 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.130076 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.130224 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":82,"threads_started":1}
I20260812 05:25:33.130504 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.122379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43206)
0812 05:25:33.122523 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.122563 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.122638 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.122823 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.122826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.123072 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.123138 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.123140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.123853 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.123854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.124443 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.124445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.124850 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.124851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.124924 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.125441 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.125450 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.127366 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.128964 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.128971 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.128974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.129183 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.129403 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.129403 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.129404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.129504 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.129876 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.129883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.129963 (+ 80us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.130066 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.130329 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":80,"thread_start_us":89,"threads_started":1}
I20260812 05:25:33.131433 31880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.125@KRBTEST.COM for oryx/127.24.66.125@KRBTEST.COM
I20260812 05:25:33.138175 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.131754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44217 (local address 127.0.0.1:45698)
0812 05:25:33.131806 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.131818 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.131865 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.131961 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.131962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.132080 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.132117 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.132119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.132859 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.132859 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.133356 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.133357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.133737 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.133738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.133796 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.134471 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.134475 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.135868 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.137256 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.137257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.137257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.137434 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.137613 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.137613 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.137614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.137665 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.137909 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.137910 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.137948 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.137998 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.138081 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 05:25:33.138226 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.131820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45698 (local address 127.24.66.125:44217)
0812 05:25:33.131854 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.131856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.131862 (+ 6us) server_negotiation.cc:407] Connection header received
0812 05:25:33.131909 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.131909 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.131917 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.131960 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.132177 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.132748 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.133434 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.133630 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.133686 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.135966 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.135967 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.135968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.135974 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.137176 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.137504 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.137504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.137505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.137555 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.137745 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.137746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.137746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.137852 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.137903 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.138022 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.138125 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":18}
I20260812 05:25:33.140010 31880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: Bootstrap starting.
I20260812 05:25:33.141258 31880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:33.141599 31880 log.cc:824] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:33.142668 31880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: No bootstrap required, opened a new log
I20260812 05:25:33.144167 31880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } }
I20260812 05:25:33.144367 31880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:33.144435 31880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63fb1e36d61646058cb8282b12a09e8e, State: Initialized, Role: FOLLOWER
I20260812 05:25:33.144596 31880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } }
I20260812 05:25:33.144909 31880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:33.144887 31888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } } }
I20260812 05:25:33.145012 31888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: This master's current role is: FOLLOWER
W20260812 05:25:33.146703 31900 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:25:33.146766 31900 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:25:33.149034 31806 master_runner.cc:428] Detected that this master 63fb1e36d61646058cb8282b12a09e8e is joining an existing cluster
I20260812 05:25:33.149121 31806 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.66.125:44217
I20260812 05:25:33.157430 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.150299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43210 (local address 127.24.66.126:45023)
0812 05:25:33.150368 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.150370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.150734 (+ 364us) server_negotiation.cc:407] Connection header received
0812 05:25:33.150813 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.150814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.150826 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.150877 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.151428 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.151972 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.152723 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.152935 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.153012 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.154298 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.154300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.154300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.154307 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.155915 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.156421 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.156423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.156423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.156497 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.156758 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.156760 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.156760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.156913 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.156976 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.157218 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.157310 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:25:33.158002 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.150587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45700 (local address 127.24.66.125:44217)
0812 05:25:33.150639 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.150642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.150739 (+ 97us) server_negotiation.cc:407] Connection header received
0812 05:25:33.150859 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.150860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.150869 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.150919 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.151479 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.152276 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.153409 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.153732 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.153775 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.154885 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.154886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.154886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.154893 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.156682 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.157181 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.157182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.157182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.157243 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.157449 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.157449 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.157450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.157591 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.157686 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.157811 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.157907 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":26}
I20260812 05:25:33.158128 31907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.150495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44217 (local address 127.0.0.1:45700)
0812 05:25:33.150671 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.150683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.150770 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.150930 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.150931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.151308 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.151384 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.151387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.152386 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.152386 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.153306 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.153309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.153849 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.153850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.153923 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.154294 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.154300 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.154771 (+ 471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.156811 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.156811 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.156812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.157049 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.157312 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.157312 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.157313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.157376 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.157676 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.157677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.157714 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.157782 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.157985 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":78,"thread_start_us":99,"threads_started":1}
I20260812 05:25:33.158646 31908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.150231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43210)
0812 05:25:33.150652 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.150662 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.150748 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.150924 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.150925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.151240 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.151327 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.151329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.152059 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.152060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.152632 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.152634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.153031 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.153032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.153101 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.153662 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.153667 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.153994 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.156001 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.156002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.156003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.156311 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.156582 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.156583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.156583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.156670 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.157034 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.157035 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.157085 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.157830 (+ 745us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.158513 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":132,"thread_start_us":96,"threads_started":1}
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.126@KRBTEST.COM for oryx/127.24.66.125@KRBTEST.COM
I20260812 05:25:33.169658 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.161420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.24.66.125:44217)
0812 05:25:33.161457 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.161459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.161714 (+ 255us) server_negotiation.cc:407] Connection header received
0812 05:25:33.161808 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.161809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.161817 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.161865 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.162462 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.162940 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.164205 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.164390 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.164454 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.166956 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.166957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.166958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.166964 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.168387 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.168892 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.168893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.168893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.168944 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.169162 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.169163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.169163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.169262 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.169312 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.169529 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.169592 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:25:33.169773 31909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.161385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44217 (local address 127.0.0.1:45702)
0812 05:25:33.161572 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.161637 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.161728 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.161871 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.161874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.162261 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.162373 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.162376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.163024 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.163025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.164094 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.164097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.164480 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.164481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.164576 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.165060 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.165066 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.166847 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.168481 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.168491 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.168494 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.168805 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.169015 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.169015 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.169016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.169093 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.169331 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.169339 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.169402 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.169504 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.169652 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
I20260812 05:25:33.170475 31315 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.66.125:44217: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e"
member_type: NON_VOTER
last_known_addr {
host: "127.24.66.125"
port: 44217
}
attrs {
promote: true
}
}
dest_uuid: "0ec15ce8ef3740b6abd2322926963037"
cas_config_opid_index: -1
I20260812 05:25:33.170878 31315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } }
I20260812 05:25:33.171445 31910 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 63fb1e36d61646058cb8282b12a09e8e (127.24.66.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } } }
I20260812 05:25:33.171774 31850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:33.172623 31910 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.66.125:44217
I20260812 05:25:33.172749 31850 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 FOLLOWER]: Refusing update from remote peer 0ec15ce8ef3740b6abd2322926963037: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 05:25:33.172821 31910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } } }
I20260812 05:25:33.172883 31910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:33.172977 31806 master.cc:537] Master@127.24.66.125:44217 shutting down...
I20260812 05:25:33.173071 31911 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 05:25:33.173941 31301 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 05:25:33.174501 31301 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 -> Peer 63fb1e36d61646058cb8282b12a09e8e (127.24.66.125:44217): Couldn't send request to peer 63fb1e36d61646058cb8282b12a09e8e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 05:25:33.176393 31806 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:33.176486 31806 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:33.176558 31806 tablet_replica.cc:330] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: stopping tablet replica
I20260812 05:25:33.189648 31806 master.cc:552] Master@127.24.66.125:44217 shutdown complete.
I20260812 05:25:33.189725 31806 master_runner.cc:305] Clearing existing system tablet
I20260812 05:25:33.189885 31806 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:33.189945 31806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:33.190874 31806 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:33.191536 31915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.191704 31806 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:33.191776 31806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:33.191804 31806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/wal
uuid: "63fb1e36d61646058cb8282b12a09e8e"
format_stamp: "Formatted at 2026-08-12 05:25:33 on dist-test-slave-zpfg"
server_key: "db3337215908f1013190bc78e362a454"
server_key_iv: "60eb96a6be9b4c15dacdedaa7487eda1"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.192895 31806 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:33.194743 31806 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 05:25:33.194844 31806 log.cc:1196] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 05:25:33.195119 31806 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: Deleting consensus metadata
I20260812 05:25:33.195283 31806 master_runner.cc:315] Copying system tablet from 127.24.66.126:45023
I20260812 05:25:33.196035 31806 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.126:45023
I20260812 05:25:33.203313 31921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.196443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43222)
0812 05:25:33.196593 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.196605 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.196672 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.196856 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.196857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.197027 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.197089 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.197090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.198151 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.198152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.198786 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.198788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.199349 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.199350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.199407 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.199933 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.199938 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.200294 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.202229 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.202230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.202230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.202480 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.202707 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.202708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.202708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.202763 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.203052 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.203052 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.203093 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.203145 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.203227 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":105,"threads_started":1}
I20260812 05:25:33.203366 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.196538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.24.66.126:45023)
0812 05:25:33.196627 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.196630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.196659 (+ 29us) server_negotiation.cc:407] Connection header received
0812 05:25:33.196732 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.196733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.196745 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.196833 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.197177 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.198056 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.198906 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.199255 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.199339 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.200385 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.200387 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.200387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.200396 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.202136 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.202554 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.202555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.202556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.202634 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.202828 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.202829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.202829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.202982 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.203061 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.203171 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.203260 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:25:33.203755 31337 tablet_copy_service.cc:140] P 0ec15ce8ef3740b6abd2322926963037: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 63fb1e36d61646058cb8282b12a09e8e ({username='oryx', principal='oryx/127.24.66.125@KRBTEST.COM'} at 127.0.0.1:43222)
I20260812 05:25:33.203910 31337 tablet_copy_service.cc:161] P 0ec15ce8ef3740b6abd2322926963037: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 63fb1e36d61646058cb8282b12a09e8e at {username='oryx', principal='oryx/127.24.66.125@KRBTEST.COM'} at 127.0.0.1:43222: session id = 63fb1e36d61646058cb8282b12a09e8e-00000000000000000000000000000000
I20260812 05:25:33.204999 31337 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 05:25:33.205852 31806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 05:25:33.207459 31806 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: tablet copy: Starting download of 0 data blocks...
I20260812 05:25:33.207643 31806 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: tablet copy: Starting download of 1 WAL segments...
I20260812 05:25:33.208818 31806 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 05:25:33.210063 31337 tablet_copy_service.cc:342] P 0ec15ce8ef3740b6abd2322926963037: Request end of tablet copy session 63fb1e36d61646058cb8282b12a09e8e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.66.125@KRBTEST.COM'} at 127.0.0.1:43222
I20260812 05:25:33.210142 31337 tablet_copy_service.cc:433] P 0ec15ce8ef3740b6abd2322926963037: ending tablet copy session 63fb1e36d61646058cb8282b12a09e8e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 63fb1e36d61646058cb8282b12a09e8e
I20260812 05:25:33.213450 31806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:33.215118 31922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:33.215118 31806 server_base.cc:1034] running on GCE node
W20260812 05:25:33.215283 31925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:33.215085 31923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:33.215454 31806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:33.215631 31806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:33.216773 31806 hybrid_clock.cc:648] HybridClock initialized: now 1786512333216749 us; error 40 us; skew 500 ppm
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:33.218190 31806 init.cc:377] Logged in from keytab as oryx/127.24.66.125@KRBTEST.COM (short username oryx)
I20260812 05:25:33.218538 31806 webserver.cc:460] Webserver started at http://127.24.66.125:34603/ using document root <none> and password file <none>
I20260812 05:25:33.218669 31806 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:33.218725 31806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:33.219369 31806 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:25:33.219959 31932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.220106 31806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 05:25:33.220155 31806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/wal
uuid: "63fb1e36d61646058cb8282b12a09e8e"
format_stamp: "Formatted at 2026-08-12 05:25:33 on dist-test-slave-zpfg"
server_key: "db3337215908f1013190bc78e362a454"
server_key_iv: "60eb96a6be9b4c15dacdedaa7487eda1"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.220214 31806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:33.230656 31806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:33.232566 31806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:33.235910 31806 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.125:44217
I20260812 05:25:33.236181 31989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.125:44217 every 8 connection(s)
I20260812 05:25:33.236331 31806 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
I20260812 05:25:33.237159 31990 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:25:33.237620 31990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: Bootstrap starting.
I20260812 05:25:33.240453 31990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:25:33.240705 31990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: Bootstrap complete.
I20260812 05:25:33.240963 31990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } }
I20260812 05:25:33.241034 31990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 63fb1e36d61646058cb8282b12a09e8e, State: Initialized, Role: LEARNER
I20260812 05:25:33.241062 31990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } }
I20260812 05:25:33.241274 31990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:33.241261 31993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } } }
I20260812 05:25:33.241381 31993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: This master's current role is: LEARNER
I20260812 05:25:33.242884 32004 catalog_manager.cc:1259] Loaded cluster ID: d3b38d4bde584a1d9c334521c2e37ce6
I20260812 05:25:33.242944 32004 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: loading cluster ID for follower catalog manager: success
I20260812 05:25:33.244210 32004 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: acquiring CA information for follower catalog manager: success
I20260812 05:25:33.244553 32004 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:25:33.291007 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.282985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45712 (local address 127.24.66.125:44217)
0812 05:25:33.283129 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.283132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.283141 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:33.283172 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.283173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.283186 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.283255 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.284036 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.284725 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.285801 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.286045 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.286101 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.287661 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.287663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.287663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.287674 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.289682 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.290143 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.290144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.290144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.290210 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.290443 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.290444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.290444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.290568 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.290652 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.290788 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.290893 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for oryx/127.24.66.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.124.keytab.
Entry for principal oryx/127.24.66.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.124.keytab.
I20260812 05:25:33.325521 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.124:46257
--webserver_interface=127.24.66.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.124.keytab
--principal=oryx/127.24.66.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:45023,127.24.66.125:44217,127.24.66.124:46257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:33.432663 32008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:33.432919 32008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:33.432950 32008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:33.434587 32008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:33.434653 32008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:33.434670 32008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:33.434685 32008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:33.434700 32008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:33.436769 32008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:45023,127.24.66.125:44217,127.24.66.124:46257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.124.keytab
--principal=oryx/127.24.66.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.124:46257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.66.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:33.437110 32008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:33.437420 32008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:33.444128 32014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:33.444120 32015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:33.444195 32017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:33.444500 32008 server_base.cc:1034] running on GCE node
I20260812 05:25:33.444788 32008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:33.445128 32008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:33.446293 32008 hybrid_clock.cc:648] HybridClock initialized: now 1786512333446245 us; error 66 us; skew 500 ppm
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:33.449544 32008 init.cc:377] Logged in from keytab as oryx/127.24.66.124@KRBTEST.COM (short username oryx)
I20260812 05:25:33.450055 32008 webserver.cc:460] Webserver started at http://127.24.66.124:44545/ using document root <none> and password file <none>
I20260812 05:25:33.450409 32008 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:33.450492 32008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:33.450659 32008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:33.451838 32008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data/instance:
uuid: "2ca4ea1990cb4214ae859a9978e75b65"
format_stamp: "Formatted at 2026-08-12 05:25:33 on dist-test-slave-zpfg"
server_key: "321d1ecceca0cbcb16566dcffaf9daf0"
server_key_iv: "8e7ba8752f755811cedc8aa53a6bd60f"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.452277 32008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/wal/instance:
uuid: "2ca4ea1990cb4214ae859a9978e75b65"
format_stamp: "Formatted at 2026-08-12 05:25:33 on dist-test-slave-zpfg"
server_key: "321d1ecceca0cbcb16566dcffaf9daf0"
server_key_iv: "8e7ba8752f755811cedc8aa53a6bd60f"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.453794 32008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 05:25:33.454859 32025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.455046 32008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:25:33.455147 32008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/wal
uuid: "2ca4ea1990cb4214ae859a9978e75b65"
format_stamp: "Formatted at 2026-08-12 05:25:33 on dist-test-slave-zpfg"
server_key: "321d1ecceca0cbcb16566dcffaf9daf0"
server_key_iv: "8e7ba8752f755811cedc8aa53a6bd60f"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.455340 32008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:33.473199 32008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:33.476115 32008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:33.476346 32008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:33.484838 32008 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.124:46257
I20260812 05:25:33.484889 32081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.124:46257 every 8 connection(s)
I20260812 05:25:33.485397 32008 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
I20260812 05:25:33.486797 32082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:33.488881 32082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:25:33.492719 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 32008
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.124@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.66.124@KRBTEST.COM
I20260812 05:25:33.500195 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.489854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43224 (local address 127.24.66.126:45023)
0812 05:25:33.489925 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.489927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.489996 (+ 69us) server_negotiation.cc:407] Connection header received
0812 05:25:33.490665 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.490666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.490679 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.490751 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.491432 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.491998 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.493184 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.493549 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.493678 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.496569 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.496571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.496571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.496580 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.498330 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.498881 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.498883 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.498883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.498952 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.499234 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.499235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.499236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.499375 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.499458 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.499957 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.500072 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:25:33.500305 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.489668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43224)
0812 05:25:33.489863 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.489908 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.490554 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.490836 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.490840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.491100 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.491344 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.491348 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.492091 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.492091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.493069 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.493072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.493709 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.493711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.493898 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.494603 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.494612 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.496447 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.498493 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.498502 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.498505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.498771 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.499014 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.499015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.499015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.499144 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.499463 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.499472 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.499682 (+ 210us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.499908 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.500079 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
I20260812 05:25:33.500967 32082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:25:33.502919 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.493384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39750 (local address 127.24.66.124:46257)
0812 05:25:33.493581 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.493587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.493612 (+ 25us) server_negotiation.cc:407] Connection header received
0812 05:25:33.493636 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.493638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.493661 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.493751 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.494438 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.495164 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.496229 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.496568 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.496653 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.499331 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.499340 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.499340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.499348 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.501522 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.501921 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.501924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.501925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.501992 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.502204 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.502205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.502205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.502467 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.502634 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.502707 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.502800 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.124@KRBTEST.COM for oryx/127.24.66.125@KRBTEST.COM
I20260812 05:25:33.509120 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.501373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.24.66.125:44217)
0812 05:25:33.501450 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.501453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.501462 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:33.501493 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.501493 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.501508 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.501588 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.502016 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.502709 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.503796 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.504044 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.504142 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.506132 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.506134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.506134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.506141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.507921 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.508262 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.508262 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.508262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.508320 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.508507 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.508507 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.508507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.508626 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.508683 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.508888 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.508969 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:25:33.509172 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.501271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44217 (local address 127.0.0.1:45720)
0812 05:25:33.501335 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.501347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.501389 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.501591 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.501592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.501744 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.501893 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.501894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.502821 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.502822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.503697 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.503699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.504140 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.504142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.504285 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.504782 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.504787 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.506042 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.508012 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.508013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.508013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.508179 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.508386 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.508387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.508387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.508441 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.508726 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.508727 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.508778 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.508865 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.508958 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 05:25:33.509706 32082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.124@KRBTEST.COM for oryx/127.24.66.124@KRBTEST.COM
I20260812 05:25:33.517904 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.510119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39764 (local address 127.24.66.124:46257)
0812 05:25:33.510178 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.510180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.510188 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:33.510418 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.510419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.510432 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.510505 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.510754 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.511344 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.512240 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.512539 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.512672 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.514949 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.514951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.514951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.514959 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.516559 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.516966 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.516967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.516968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.517033 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.517261 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.517262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.517262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.517389 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.517469 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.517634 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.517777 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":42}
I20260812 05:25:33.517968 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.510011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:46257 (local address 127.0.0.1:39764)
0812 05:25:33.510081 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.510093 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.510218 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.510506 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.510507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.510648 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.510681 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.510683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.511428 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.511430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.512099 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.512102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.512642 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.512643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.512695 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.513349 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.513354 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.514847 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.516686 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.516687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.516687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.516880 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.517108 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.517109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.517109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.517177 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.517471 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.517472 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.517514 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.517609 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.517836 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":124}
I20260812 05:25:33.519977 32082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: Bootstrap starting.
I20260812 05:25:33.521261 32082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:33.521704 32082 log.cc:824] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:33.522809 32082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: No bootstrap required, opened a new log
I20260812 05:25:33.525259 32082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } }
I20260812 05:25:33.525449 32082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:33.525506 32082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca4ea1990cb4214ae859a9978e75b65, State: Initialized, Role: FOLLOWER
I20260812 05:25:33.525699 32082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } }
I20260812 05:25:33.526162 32082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:33.526222 32088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } } }
I20260812 05:25:33.526460 32088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 05:25:33.528673 32100 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:25:33.528751 32100 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:25:33.531672 32008 master_runner.cc:428] Detected that this master 2ca4ea1990cb4214ae859a9978e75b65 is joining an existing cluster
I20260812 05:25:33.531721 32008 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.66.124:46257
I20260812 05:25:33.554322 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 31381
I20260812 05:25:33.556381 32108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.533460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43238)
0812 05:25:33.534276 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.534299 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.534422 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.534623 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.534624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.535019 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.535186 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.535188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.536200 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.536201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.537058 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.537061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.537599 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.537600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.537729 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.538227 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.538230 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.551394 (+ 13164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.553770 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.553771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.553771 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.553977 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.554215 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.554215 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.554216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.554264 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.555951 (+ 1687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.555952 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.555999 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.556056 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.556213 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":194,"threads_started":1}
I20260812 05:25:33.556727 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.533460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.24.66.126:45023)
0812 05:25:33.533520 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.533522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.534442 (+ 920us) server_negotiation.cc:407] Connection header received
0812 05:25:33.534481 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.534482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.534496 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.534566 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.535296 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.536084 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.537155 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.537472 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.537542 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.551545 (+ 14003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.551547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.551548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.551560 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.553664 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.554060 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.554062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.554062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.554143 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.554332 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.554333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.554333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.555843 (+ 1510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.555920 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.556512 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.556599 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 05:25:33.556973 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.533397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39766 (local address 127.24.66.124:46257)
0812 05:25:33.533441 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.533443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.533725 (+ 282us) server_negotiation.cc:407] Connection header received
0812 05:25:33.533799 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.533800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.533809 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.533855 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.534531 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.535013 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.536256 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.536543 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.536585 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.538009 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.538011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.538011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.538017 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.553463 (+ 15446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.553899 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.553900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.553901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.553950 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.554134 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.554135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.554135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.554288 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.554332 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.556850 (+ 2518us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.556923 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:25:33.557220 32106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.533458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:46257 (local address 127.0.0.1:39766)
0812 05:25:33.533641 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.533652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.533742 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.533868 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.533869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.534115 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.534444 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.534446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.535097 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.535098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.536159 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.536162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.536738 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.536738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.536815 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.537379 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.537384 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.537944 (+ 560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.553579 (+ 15635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.553581 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.553581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.553821 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.554015 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.554015 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.554016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.554068 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.554399 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.554400 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.554441 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.557062 (+ 2621us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.557158 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":132,"threads_started":1}
I20260812 05:25:33.561741 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.533939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45724 (local address 127.24.66.125:44217)
0812 05:25:33.534025 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.534027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.535384 (+ 1357us) server_negotiation.cc:407] Connection header received
0812 05:25:33.535493 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.535494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.535508 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.535543 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.536252 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.537121 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.538514 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.538798 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.538867 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.551526 (+ 12659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.551528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.551528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.551539 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.553559 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.560766 (+ 7207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.560768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.560769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.560848 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.561070 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.561071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.561072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.561242 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.561323 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.561550 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.561650 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 05:25:33.562172 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.534062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44217 (local address 127.0.0.1:45724)
0812 05:25:33.535321 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.535332 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.535413 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.535615 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.535616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.536027 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.536135 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.536138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.537240 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.537240 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.538333 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.538376 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.538926 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.538927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.539001 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.539422 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.539427 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.551352 (+ 11925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.560430 (+ 9078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.560431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.560431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.560644 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.560927 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.560928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.560929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.560997 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.561367 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.561368 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.561423 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.561829 (+ 406us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.561966 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1241,"thread_start_us":132,"threads_started":1}
I20260812 05:25:33.565699 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:42989
--local_ip_for_outbound_sockets=127.24.66.65
--tserver_master_addrs=127.24.66.126:45023
--webserver_port=33059
--webserver_interface=127.24.66.65
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab
--principal=oryx/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.66.126:45023,127.24.66.125:44217
--tserver_master_addrs=127.24.66.126:45023,127.24.66.125:44217,127.24.66.124:46257 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512332, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.126@KRBTEST.COM for oryx/127.24.66.124@KRBTEST.COM
I20260812 05:25:33.571839 31909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.563747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:46257 (local address 127.0.0.1:39776)
0812 05:25:33.563804 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.563815 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.563854 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.564058 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.564059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.564270 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.564332 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.564335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.565218 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.565220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.566029 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.566033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.566584 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.566586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.566691 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.567198 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.567205 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.568892 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.570717 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.570718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.570719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.570919 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.571141 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.571142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.571143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.571208 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.571479 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.571480 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.571532 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.571625 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.571741 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 05:25:33.571838 32085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.563776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.24.66.124:46257)
0812 05:25:33.563919 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.563920 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.563926 (+ 6us) server_negotiation.cc:407] Connection header received
0812 05:25:33.563953 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.563954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.563962 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.564048 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.564421 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.565119 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.566134 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.566495 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.566565 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.568997 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.568998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.568999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.569007 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.570643 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.571006 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.571007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.571008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.571065 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.571285 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.571286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.571286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.571411 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.571487 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.571655 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.571739 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260812 05:25:33.572481 31315 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.66.124:46257: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65"
member_type: NON_VOTER
last_known_addr {
host: "127.24.66.124"
port: 46257
}
attrs {
promote: true
}
}
dest_uuid: "0ec15ce8ef3740b6abd2322926963037"
cas_config_opid_index: 5
I20260812 05:25:33.572783 31315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } }
I20260812 05:25:33.573423 31910 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2ca4ea1990cb4214ae859a9978e75b65 (127.24.66.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:33.574299 31910 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.66.124:46257
I20260812 05:25:33.574702 31911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:33.574817 31911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:33.575090 32052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:33.576409 32052 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 FOLLOWER]: Refusing update from remote peer 0ec15ce8ef3740b6abd2322926963037: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 05:25:33.576787 32008 master.cc:537] Master@127.24.66.124:46257 shutting down...
I20260812 05:25:33.576786 31911 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 05:25:33.577625 31299 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 -> Peer 2ca4ea1990cb4214ae859a9978e75b65 (127.24.66.124:46257): Couldn't send request to peer 2ca4ea1990cb4214ae859a9978e75b65. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 05:25:33.580153 32008 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:33.580253 32008 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:33.580299 32008 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: stopping tablet replica
I20260812 05:25:33.583410 31909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.573311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44217 (local address 127.0.0.1:45726)
0812 05:25:33.577398 (+ 4087us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.577412 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.577742 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.577996 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.577997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.577998 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:33.578052 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.578054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.580095 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.580097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.581876 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.581881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.583090 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.583091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.583166 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.583198 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.583258 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.583313 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4067}
I20260812 05:25:33.583729 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.573373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.24.66.125:44217)
0812 05:25:33.573418 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.573420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.577705 (+ 4285us) server_negotiation.cc:407] Connection header received
0812 05:25:33.577830 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.577831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.577860 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.577929 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:33.578132 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.579969 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.582083 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.582772 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.582839 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.583050 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.583577 (+ 527us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.583641 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:25:33.583990 31960 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 LEARNER]: Refusing update from remote peer 0ec15ce8ef3740b6abd2322926963037: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 05:25:33.584237 31911 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 05:25:33.584683 32008 master.cc:552] Master@127.24.66.124:46257 shutdown complete.
I20260812 05:25:33.584767 32008 master_runner.cc:305] Clearing existing system tablet
I20260812 05:25:33.584949 32008 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:33.585035 32008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:33.584893 31960 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2ca4ea1990cb4214ae859a9978e75b65 (127.24.66.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:33.585806 32008 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.586153 31993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ec15ce8ef3740b6abd2322926963037. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:33.586253 31993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: This master's current role is: LEARNER
I20260812 05:25:33.586295 31993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: NON_VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: true } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:33.586443 31993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: This master's current role is: LEARNER
I20260812 05:25:33.586555 32113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.586689 32008 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:33.586738 32008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:25:33.586762 32008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/wal
uuid: "2ca4ea1990cb4214ae859a9978e75b65"
format_stamp: "Formatted at 2026-08-12 05:25:33 on dist-test-slave-zpfg"
server_key: "321d1ecceca0cbcb16566dcffaf9daf0"
server_key_iv: "8e7ba8752f755811cedc8aa53a6bd60f"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.587056 31911 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037: attempting to promote NON_VOTER 63fb1e36d61646058cb8282b12a09e8e to VOTER
I20260812 05:25:33.587280 31911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } }
I20260812 05:25:33.587626 32008 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:33.588034 31960 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 LEARNER]: Refusing update from remote peer 0ec15ce8ef3740b6abd2322926963037: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 05:25:33.588200 31910 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 05:25:33.588713 31909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.587744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:46257 (local address 127.0.0.1:39788)
0812 05:25:33.588622 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.588681 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":846}
I20260812 05:25:33.589030 31909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.588957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:46257 (local address 127.0.0.1:39794)
0812 05:25:33.588987 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.589007 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 05:25:33.589083 31299 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 -> Peer 2ca4ea1990cb4214ae859a9978e75b65 (127.24.66.124:46257): Couldn't send request to peer 2ca4ea1990cb4214ae859a9978e75b65. Status: Network error: Client connection negotiation failed: client connection to 127.24.66.124:46257: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 05:25:33.589440 31910 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 63fb1e36d61646058cb8282b12a09e8e (127.24.66.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:33.590289 31910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:33.590402 31910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:33.590466 31910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:33.590523 31910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:33.590677 32008 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 05:25:33.590749 32008 log.cc:1196] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 05:25:33.590718 31960 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 63fb1e36d61646058cb8282b12a09e8e (127.24.66.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:33.590999 32008 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: Deleting consensus metadata
I20260812 05:25:33.591127 32008 master_runner.cc:315] Copying system tablet from 127.24.66.126:45023
I20260812 05:25:33.591785 32008 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.126:45023
I20260812 05:25:33.591845 32112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:33.591936 32112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:25:33.598438 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.592199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.24.66.126:45023)
0812 05:25:33.592240 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.592242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.592526 (+ 284us) server_negotiation.cc:407] Connection header received
0812 05:25:33.592559 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.592560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.592571 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.592635 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.592972 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.593479 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.594246 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.594538 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.594618 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.595652 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.595654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.595654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.595662 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.597182 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.597555 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.597556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.597557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.597625 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.597847 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.597848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.597848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.597973 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.598054 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.598222 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.598305 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:25:33.598769 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.592311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43248)
0812 05:25:33.592437 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.592448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.592512 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.592641 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.592642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.592831 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.592881 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.592882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.593566 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.593567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.594147 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.594149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.594622 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.594624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.594695 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.595226 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.595231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.595552 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.597289 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.597291 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.597292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.597480 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.597698 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.597699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.597699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.597779 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.598045 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.598046 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.598094 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.598189 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.598312 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260812 05:25:33.599270 31337 tablet_copy_service.cc:140] P 0ec15ce8ef3740b6abd2322926963037: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2ca4ea1990cb4214ae859a9978e75b65 ({username='oryx', principal='oryx/127.24.66.124@KRBTEST.COM'} at 127.0.0.1:43248)
I20260812 05:25:33.599356 31337 tablet_copy_service.cc:161] P 0ec15ce8ef3740b6abd2322926963037: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2ca4ea1990cb4214ae859a9978e75b65 at {username='oryx', principal='oryx/127.24.66.124@KRBTEST.COM'} at 127.0.0.1:43248: session id = 2ca4ea1990cb4214ae859a9978e75b65-00000000000000000000000000000000
I20260812 05:25:33.599980 31337 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 05:25:33.600530 32008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 05:25:33.601543 32008 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: tablet copy: Starting download of 0 data blocks...
I20260812 05:25:33.601724 32008 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: tablet copy: Starting download of 1 WAL segments...
I20260812 05:25:33.602607 32008 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 05:25:33.603399 31337 tablet_copy_service.cc:342] P 0ec15ce8ef3740b6abd2322926963037: Request end of tablet copy session 2ca4ea1990cb4214ae859a9978e75b65-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.66.124@KRBTEST.COM'} at 127.0.0.1:43248
I20260812 05:25:33.603459 31337 tablet_copy_service.cc:433] P 0ec15ce8ef3740b6abd2322926963037: ending tablet copy session 2ca4ea1990cb4214ae859a9978e75b65-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2ca4ea1990cb4214ae859a9978e75b65
I20260812 05:25:33.605883 32008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:33.607203 32121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:33.607300 32124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:33.607316 32122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:33.607550 32008 server_base.cc:1034] running on GCE node
I20260812 05:25:33.607686 32008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:33.607874 32008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:33.609012 32008 hybrid_clock.cc:648] HybridClock initialized: now 1786512333609010 us; error 24 us; skew 500 ppm
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:33.610894 32008 init.cc:377] Logged in from keytab as oryx/127.24.66.124@KRBTEST.COM (short username oryx)
I20260812 05:25:33.611227 32008 webserver.cc:460] Webserver started at http://127.24.66.124:46813/ using document root <none> and password file <none>
I20260812 05:25:33.611349 32008 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:33.611387 32008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:33.612057 32008 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.612665 32131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.612776 32008 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.612814 32008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/wal
uuid: "2ca4ea1990cb4214ae859a9978e75b65"
format_stamp: "Formatted at 2026-08-12 05:25:33 on dist-test-slave-zpfg"
server_key: "321d1ecceca0cbcb16566dcffaf9daf0"
server_key_iv: "8e7ba8752f755811cedc8aa53a6bd60f"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.612879 32008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:33.618021 32008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:33.620059 32008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:33.623423 32008 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.124:46257
I20260812 05:25:33.623808 32008 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
I20260812 05:25:33.624696 32188 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:25:33.625128 32188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: Bootstrap starting.
I20260812 05:25:33.626360 32187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.124:46257 every 8 connection(s)
I20260812 05:25:33.627584 32188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:25:33.627907 32188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: Bootstrap complete.
I20260812 05:25:33.628125 32188 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } }
I20260812 05:25:33.628221 32188 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2ca4ea1990cb4214ae859a9978e75b65, State: Initialized, Role: LEARNER
I20260812 05:25:33.628268 32188 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } }
I20260812 05:25:33.628453 32188 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:33.628446 32192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:33.628683 32192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: This master's current role is: LEARNER
I20260812 05:25:33.629958 32204 catalog_manager.cc:1259] Loaded cluster ID: d3b38d4bde584a1d9c334521c2e37ce6
I20260812 05:25:33.630046 32204 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: loading cluster ID for follower catalog manager: success
I20260812 05:25:33.631385 32204 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: acquiring CA information for follower catalog manager: success
I20260812 05:25:33.631692 32204 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 05:25:33.692307 32109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:33.692632 32109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:33.692673 32109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:33.694656 32109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:33.694723 32109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:33.694770 32109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:33.697216 32109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab
--principal=oryx/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:42989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=33059
--tserver_master_addrs=127.24.66.126:45023,127.24.66.125:44217,127.24.66.124:46257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:33.697564 32109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:33.697934 32109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:33.703857 32209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:33.703958 32212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:33.703867 32210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:33.704342 32109 server_base.cc:1034] running on GCE node
I20260812 05:25:33.704571 32109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:33.704834 32109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:33.705999 32109 hybrid_clock.cc:648] HybridClock initialized: now 1786512333705984 us; error 33 us; skew 500 ppm
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:33.709235 32109 init.cc:377] Logged in from keytab as oryx/127.24.66.65@KRBTEST.COM (short username oryx)
I20260812 05:25:33.709856 32109 webserver.cc:460] Webserver started at http://127.24.66.65:33059/ using document root <none> and password file <none>
I20260812 05:25:33.710120 32109 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:33.710210 32109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:33.712469 32109 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:25:33.713287 32220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.713498 32109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:33.713654 32109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "044a7521d21e4c0b93be61cd66d6bea3"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "caca04f4544b04a444891e79639d08e2"
server_key_iv: "ff6e20eab998e6714ad873efc32eda36"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.714138 32109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:33.739022 32109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:33.742189 32109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:33.742403 32109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:33.742849 32109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:33.743254 32109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:33.743299 32109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.743330 32109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:33.743355 32109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.756778 32109 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:42989
I20260812 05:25:33.756821 32338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:42989 every 8 connection(s)
I20260812 05:25:33.757230 32109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:33.757817 31658 heartbeater.cc:503] Master 127.24.66.126:45023 was elected leader, sending a full tablet report...
I20260812 05:25:33.764988 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 32109
I20260812 05:25:33.765218 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 31521
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.125@KRBTEST.COM
I20260812 05:25:33.773365 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.758606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:33927 (local address 127.24.66.126:45023)
0812 05:25:33.758681 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.758683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.759149 (+ 466us) server_negotiation.cc:407] Connection header received
0812 05:25:33.760521 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.760523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.760537 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.760570 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.761606 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.762446 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.764296 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.764484 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.764564 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.769050 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.769053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.769053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.769061 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.770997 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.771840 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.771842 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.771843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.771926 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.772356 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.772357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.772358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.772533 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.772612 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.773116 (+ 504us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.773213 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:25:33.773896 32347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.758483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.65:33927)
0812 05:25:33.759058 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.759069 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.760382 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.760703 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.760704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.761183 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.761483 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.761486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.762527 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.762528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.763772 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.763774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.764566 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.764567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.764682 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.766902 (+ 2220us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.766913 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.768905 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.771109 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.771121 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.771125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.771708 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.772018 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.772019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.772019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.772230 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.772620 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.772627 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.772870 (+ 243us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.773070 (+ 200us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.773549 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":32,"thread_start_us":366,"threads_started":1}
I20260812 05:25:33.776904 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.758111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44217 (local address 127.24.66.65:49567)
0812 05:25:33.758397 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.758619 (+ 222us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.760350 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.760686 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.760690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.761124 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.761470 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.761475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.762323 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.762324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.763772 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.763774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.764391 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.764393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.764495 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.771618 (+ 7123us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.771623 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.773223 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.775584 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.775585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.775586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.775828 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.776125 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.776126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.776126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.776195 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.776574 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.776574 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.776617 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.776669 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.776822 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":6,"thread_start_us":125,"threads_started":1}
I20260812 05:25:33.776954 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.758504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:49567 (local address 127.24.66.125:44217)
0812 05:25:33.758558 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.758561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.758706 (+ 145us) server_negotiation.cc:407] Connection header received
0812 05:25:33.760498 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.760499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.760515 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.760601 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.761587 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.762112 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.763864 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.764181 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.764244 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.773349 (+ 9105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.773351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.773352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.773362 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.775479 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.775930 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.775931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.775931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.776008 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.776281 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.776283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.776283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.776472 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.776553 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.776751 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.776848 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.124@KRBTEST.COM
I20260812 05:25:33.781684 32342 heartbeater.cc:347] Connected to a master server at 127.24.66.125:44217
I20260812 05:25:33.781829 32342 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:33.781911 32341 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45023
I20260812 05:25:33.781970 32341 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:33.782064 32342 heartbeater.cc:511] Master 127.24.66.125:44217 requested a full tablet report, sending...
I20260812 05:25:33.782450 32341 heartbeater.cc:511] Master 127.24.66.126:45023 requested a full tablet report, sending...
I20260812 05:25:33.782681 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:38509
--local_ip_for_outbound_sockets=127.24.66.66
--tserver_master_addrs=127.24.66.126:45023
--webserver_port=40657
--webserver_interface=127.24.66.66
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab
--principal=oryx/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.66.126:45023,127.24.66.125:44217
--tserver_master_addrs=127.24.66.126:45023,127.24.66.125:44217,127.24.66.124:46257 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:25:33.782918 31315 ts_manager.cc:194] Re-registered known tserver with Master: 044a7521d21e4c0b93be61cd66d6bea3 (127.24.66.65:42989)
I20260812 05:25:33.782981 31949 ts_manager.cc:194] Registered new tserver with Master: 044a7521d21e4c0b93be61cd66d6bea3 (127.24.66.65:42989)
I20260812 05:25:33.783465 31315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:33927
I20260812 05:25:33.784454 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.758684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:37829 (local address 127.24.66.124:46257)
0812 05:25:33.758805 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.758808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.758818 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:33.760557 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.760559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.760572 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.760631 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.761614 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.762663 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.763714 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.764035 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.764116 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.780745 (+ 16629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.780748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.780748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.780758 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.783092 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.783575 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.783576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.783577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.783646 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.783882 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.783883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.783883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.784035 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.784081 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.784301 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.784385 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260812 05:25:33.784615 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.758183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:46257 (local address 127.24.66.65:37829)
0812 05:25:33.758686 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.758698 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.760382 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.760703 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.760704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.761313 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.761515 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.761517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.762745 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.762746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.763600 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.763602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.764144 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.764145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.764486 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.779024 (+ 14538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.779030 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.780623 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.783259 (+ 2636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.783260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.783260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.783472 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.783739 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.783740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.783740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.783808 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.784128 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.784129 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.784175 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.784245 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.784397 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":3,"thread_start_us":208,"threads_started":1}
I20260812 05:25:33.785012 32340 heartbeater.cc:347] Connected to a master server at 127.24.66.124:46257
I20260812 05:25:33.785073 32340 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:33.785162 32340 heartbeater.cc:511] Master 127.24.66.124:46257 requested a full tablet report, sending...
I20260812 05:25:33.785611 32142 ts_manager.cc:194] Registered new tserver with Master: 044a7521d21e4c0b93be61cd66d6bea3 (127.24.66.65:42989)
W20260812 05:25:33.873577 32349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:33.873804 32349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:33.873832 32349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:33.875133 32349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:33.875183 32349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:33.875227 32349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:33.876857 32349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab
--principal=oryx/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:38509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=40657
--tserver_master_addrs=127.24.66.126:45023,127.24.66.125:44217,127.24.66.124:46257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:33.877139 32349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:33.877362 32349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:33.883296 32355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:33.883400 32357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:33.883307 32354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:33.883602 32349 server_base.cc:1034] running on GCE node
I20260812 05:25:33.883824 32349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:33.884097 32349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:33.885283 32349 hybrid_clock.cc:648] HybridClock initialized: now 1786512333885250 us; error 53 us; skew 500 ppm
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:33.892182 32349 init.cc:377] Logged in from keytab as oryx/127.24.66.66@KRBTEST.COM (short username oryx)
I20260812 05:25:33.892787 32349 webserver.cc:460] Webserver started at http://127.24.66.66:40657/ using document root <none> and password file <none>
I20260812 05:25:33.893054 32349 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:33.893147 32349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:33.894683 32349 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 05:25:33.895432 32364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.895745 32349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:33.895918 32349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "9e05066fbe2f217e37d4c4a80966555f"
server_key_iv: "c38a07cdacbbe1f4c592d6116763e323"
server_key_version: "encryptionkey@0"
I20260812 05:25:33.896312 32349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:33.913205 32349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:33.915747 32349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:33.915933 32349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:33.916169 32349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:33.916507 32349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:33.916541 32349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.916564 32349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:33.916579 32349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:33.923606 32349 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:38509
I20260812 05:25:33.923707 32481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:38509 every 8 connection(s)
I20260812 05:25:33.924063 32349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:33.930073 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 32349
I20260812 05:25:33.930263 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 31663
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.125@KRBTEST.COM
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.124@KRBTEST.COM
Aug 12 05:25:33 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512333, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:33.937222 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.925333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:50029 (local address 127.24.66.125:44217)
0812 05:25:33.925381 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.925384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.925501 (+ 117us) server_negotiation.cc:407] Connection header received
0812 05:25:33.925917 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.925918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.925933 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.925967 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.926799 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.927622 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.928996 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.929360 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.929446 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.933530 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.933532 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.933532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.933541 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.935390 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.936036 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.936037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.936037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.936115 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.936375 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.936376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.936376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.936534 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.936579 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.937026 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.937125 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:25:33.937635 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.924940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:33341 (local address 127.24.66.124:46257)
0812 05:25:33.925373 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.925376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.925390 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:25:33.925887 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.925889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.925902 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.925949 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.926785 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.927311 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.928425 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.928731 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.928841 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.933022 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.933024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.933025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.933035 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.935161 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.935898 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.935900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.935900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.935972 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.936384 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.936385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.936385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.936586 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.936640 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.937147 (+ 507us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.937284 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397}
I20260812 05:25:33.937793 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.924948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:43445 (local address 127.24.66.126:45023)
0812 05:25:33.925000 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:33.925002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:33.925434 (+ 432us) server_negotiation.cc:407] Connection header received
0812 05:25:33.925917 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:33.925919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:33.925933 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:33.925967 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:33.926800 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.927575 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.928976 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.929332 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.929378 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.934367 (+ 4989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:33.934370 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:33.934370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:33.934380 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:33.936446 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.936819 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.936821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.936821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.936881 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:33.937091 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:33.937092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:33.937092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:33.937220 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:33.937265 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:33.937632 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 05:25:33.937713 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:25:33.937922 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.925241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44217 (local address 127.24.66.66:50029)
0812 05:25:33.925454 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.925463 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.925792 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.926494 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.926495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.926644 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.926700 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.926701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.927735 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.927736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.928912 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.928914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.929444 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.929445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.929516 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.930024 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.930029 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.933404 (+ 3375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.935489 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.935490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.935490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.935935 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.936199 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.936199 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.936200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.936296 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.936628 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.936635 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.936855 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.937343 (+ 488us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.937572 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":9,"thread_start_us":65,"threads_started":1}
I20260812 05:25:33.938515 32488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.924889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.66:43445)
0812 05:25:33.925367 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.925379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.925792 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.926068 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.926070 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.926471 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.926682 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.926689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.927705 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.927706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.928840 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.928843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.929463 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.929464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.929542 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.931944 (+ 2402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.931949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.933719 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.936526 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.936527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.936527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.936732 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.936958 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.936959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.936960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.937016 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.937457 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.937457 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.937500 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.938191 (+ 691us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.938299 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":67,"thread_start_us":91,"threads_started":1}
I20260812 05:25:33.939081 32487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:33.924889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:46257 (local address 127.24.66.66:33341)
0812 05:25:33.925116 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:33.925155 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:33.925759 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:33.926044 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:33.926047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:33.926469 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:33.926681 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.926685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.927428 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.927430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.928330 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:33.928332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:33.928854 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:33.928855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:33.929036 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:33.929863 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:33.929874 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:33.932890 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:33.935288 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.935301 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.935304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.935783 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.936062 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:33.936063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:33.936064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:33.936286 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:33.936699 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:33.936700 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:33.936856 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 05:25:33.937784 (+ 928us) client_negotiation.cc:240] Negotiation successful
0812 05:25:33.937925 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
I20260812 05:25:33.943027 32482 heartbeater.cc:347] Connected to a master server at 127.24.66.124:46257
I20260812 05:25:33.943166 32482 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:33.943410 32482 heartbeater.cc:511] Master 127.24.66.124:46257 requested a full tablet report, sending...
I20260812 05:25:33.943745 32483 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45023
I20260812 05:25:33.943792 32483 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:33.943818 32484 heartbeater.cc:347] Connected to a master server at 127.24.66.125:44217
I20260812 05:25:33.943861 32483 heartbeater.cc:511] Master 127.24.66.126:45023 requested a full tablet report, sending...
I20260812 05:25:33.943871 32484 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:33.944000 32484 heartbeater.cc:511] Master 127.24.66.125:44217 requested a full tablet report, sending...
I20260812 05:25:33.944052 32142 ts_manager.cc:194] Registered new tserver with Master: 7e191f23bc0c4e04aa4ec9cc21fbf4f4 (127.24.66.66:38509)
I20260812 05:25:33.944401 31315 ts_manager.cc:194] Re-registered known tserver with Master: 7e191f23bc0c4e04aa4ec9cc21fbf4f4 (127.24.66.66:38509)
I20260812 05:25:33.944459 31949 ts_manager.cc:194] Registered new tserver with Master: 7e191f23bc0c4e04aa4ec9cc21fbf4f4 (127.24.66.66:38509)
I20260812 05:25:33.944897 31315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:43445
I20260812 05:25:33.947563 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:34223
--local_ip_for_outbound_sockets=127.24.66.67
--tserver_master_addrs=127.24.66.126:45023
--webserver_port=45111
--webserver_interface=127.24.66.67
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab
--principal=oryx/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.66.126:45023,127.24.66.125:44217
--tserver_master_addrs=127.24.66.126:45023,127.24.66.125:44217,127.24.66.124:46257 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:25:34.021770 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.018511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.24.66.124:46257)
0812 05:25:34.018588 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.018643 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.018654 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:34.018685 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.018686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.018717 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.018791 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:34.018949 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.019493 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.020984 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.021390 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.021455 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.021503 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.021675 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.021711 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:25:34.021811 31909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.018447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:46257 (local address 127.0.0.1:39808)
0812 05:25:34.018514 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:34.018551 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:34.018615 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:34.018797 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:34.018798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:34.018800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:34.018878 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.018880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.019642 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.019645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.020865 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.020871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.021497 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.021499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.021578 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.021585 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 05:25:34.021661 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 05:25:34.021715 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 05:25:34.022167 32152 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 05:25:34.022472 32192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ec15ce8ef3740b6abd2322926963037. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:34.022589 32192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: This master's current role is: LEARNER
W20260812 05:25:34.037918 32489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:34.038161 32489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:34.038187 32489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:34.039877 32489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:34.039938 32489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:34.039968 32489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:34.041690 32489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab
--principal=oryx/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:34223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=45111
--tserver_master_addrs=127.24.66.126:45023,127.24.66.125:44217,127.24.66.124:46257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:34.042016 32489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:34.042275 32489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:34.048528 32497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:34.048595 32489 server_base.cc:1034] running on GCE node
W20260812 05:25:34.048545 32494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:34.048522 32495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:34.049073 32489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:34.049335 32489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:34.050479 32489 hybrid_clock.cc:648] HybridClock initialized: now 1786512334050452 us; error 43 us; skew 500 ppm
Aug 12 05:25:34 dist-test-slave-zpfg krb5kdc[31267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512334, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:34.053143 32489 init.cc:377] Logged in from keytab as oryx/127.24.66.67@KRBTEST.COM (short username oryx)
I20260812 05:25:34.053601 32489 webserver.cc:460] Webserver started at http://127.24.66.67:45111/ using document root <none> and password file <none>
I20260812 05:25:34.053802 32489 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:34.053867 32489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:34.055341 32489 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:34.055938 32505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:34.056118 32489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:34.056216 32489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "86dc52beff2747f9bbe8f6041061d1a1"
format_stamp: "Formatted at 2026-08-12 05:25:32 on dist-test-slave-zpfg"
server_key: "d38675db716a712eb43ffc36fdbaab13"
server_key_iv: "9daff91392a8f47da9905d9df50b8ed2"
server_key_version: "encryptionkey@0"
I20260812 05:25:34.056607 32489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:34.074517 32489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:34.077206 32489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:34.077390 32489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:34.077677 32489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:34.078012 32489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:34.078050 32489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:34.078074 32489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:34.078087 32489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:34.084703 32489 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:34223
I20260812 05:25:34.084842 32623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:34223 every 8 connection(s)
I20260812 05:25:34.085233 32489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
Aug 12 05:25:34 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512334, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.125@KRBTEST.COM
Aug 12 05:25:34 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512334, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:34.093860 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 32489
Aug 12 05:25:34 dist-test-slave-zpfg krb5kdc[31267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512334, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.124@KRBTEST.COM
I20260812 05:25:34.098028 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.086100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:49171 (local address 127.24.66.126:45023)
0812 05:25:34.086155 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.086157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.086430 (+ 273us) server_negotiation.cc:407] Connection header received
0812 05:25:34.087703 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.087704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.087717 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.087763 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:34.088701 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.089229 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.090309 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.090576 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.090657 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.094178 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:34.094179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:34.094180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:34.094190 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:34.096170 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.096732 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.096733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.096733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.096809 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.097147 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.097148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.097148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.097334 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:34.097384 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.097802 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.097891 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:25:34.098095 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.086065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:52627 (local address 127.24.66.124:46257)
0812 05:25:34.086130 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.086133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.086485 (+ 352us) server_negotiation.cc:407] Connection header received
0812 05:25:34.087647 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.087648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.087664 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.087731 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:34.088740 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.089488 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.091012 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.091320 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.091390 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.094545 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:34.094546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:34.094547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:34.094556 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:34.096798 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.097224 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.097226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.097226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.097286 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.097509 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.097511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.097511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.097643 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:34.097699 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.097908 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.097999 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:25:34.098095 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.086782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:41781 (local address 127.24.66.125:44217)
0812 05:25:34.086860 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.086862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.086873 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:34.087673 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.087674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.087688 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.087721 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:34.088704 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.089553 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.090876 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.091166 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.091261 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.094545 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:34.094546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:34.094547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:34.094556 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:34.096685 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.097266 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.097267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.097267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.097321 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.097520 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.097521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.097521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.097643 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:34.097688 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.097918 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.097999 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:25:34.098747 32628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.086039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.67:49171)
0812 05:25:34.086270 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:34.086309 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:34.087488 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:34.087857 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:34.087861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:34.088322 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:34.088589 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.088594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.089355 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.089356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.090199 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.090201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.090667 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.090667 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.090824 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.091455 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:34.091466 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:34.094030 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:34.096274 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:34.096283 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:34.096286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:34.096633 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:34.096895 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:34.096896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:34.096896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:34.097055 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:34.097439 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:34.097448 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:34.097632 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 05:25:34.098241 (+ 609us) client_negotiation.cc:240] Negotiation successful
0812 05:25:34.098467 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":3,"thread_start_us":133,"threads_started":1}
I20260812 05:25:34.098789 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.086519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44217 (local address 127.24.66.67:41781)
0812 05:25:34.086822 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:34.086834 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:34.087548 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:34.087858 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:34.087861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:34.088322 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:34.088591 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.088594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.089647 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.089648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.090783 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.090786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.091287 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.091289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.091371 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.091740 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:34.091744 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:34.094429 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:34.096788 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:34.096790 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:34.096790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:34.097074 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:34.097396 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:34.097396 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:34.097397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:34.097445 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:34.097734 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:34.097735 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:34.097786 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 05:25:34.098197 (+ 411us) client_negotiation.cc:240] Negotiation successful
0812 05:25:34.098571 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":144,"thread_start_us":134,"threads_started":1}
I20260812 05:25:34.099247 32625 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45023
I20260812 05:25:34.099352 32625 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:34.099578 32626 heartbeater.cc:347] Connected to a master server at 127.24.66.125:44217
I20260812 05:25:34.099596 32625 heartbeater.cc:511] Master 127.24.66.126:45023 requested a full tablet report, sending...
I20260812 05:25:34.099776 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.086040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:46257 (local address 127.24.66.67:52627)
0812 05:25:34.086426 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:34.086436 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:34.087488 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:34.087858 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:34.087861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:34.088322 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:34.088597 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.088598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.089686 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.089687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.090919 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.090921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.091850 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.091851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.091925 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.092363 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:34.092369 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:34.094429 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:34.096880 (+ 2451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:34.096881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:34.096881 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:34.097136 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:34.097372 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:34.097373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:34.097374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:34.097434 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:34.097733 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:34.097734 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:34.097786 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 05:25:34.099417 (+ 1631us) client_negotiation.cc:240] Negotiation successful
0812 05:25:34.099539 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":72,"thread_start_us":91,"threads_started":1}
I20260812 05:25:34.099642 32626 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:34.099913 32626 heartbeater.cc:511] Master 127.24.66.125:44217 requested a full tablet report, sending...
I20260812 05:25:34.100239 32624 heartbeater.cc:347] Connected to a master server at 127.24.66.124:46257
I20260812 05:25:34.100288 32624 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:34.100284 31949 ts_manager.cc:194] Registered new tserver with Master: 86dc52beff2747f9bbe8f6041061d1a1 (127.24.66.67:34223)
I20260812 05:25:34.100287 31315 ts_manager.cc:194] Re-registered known tserver with Master: 86dc52beff2747f9bbe8f6041061d1a1 (127.24.66.67:34223)
I20260812 05:25:34.100365 32624 heartbeater.cc:511] Master 127.24.66.124:46257 requested a full tablet report, sending...
I20260812 05:25:34.100663 32142 ts_manager.cc:194] Registered new tserver with Master: 86dc52beff2747f9bbe8f6041061d1a1 (127.24.66.67:34223)
I20260812 05:25:34.100762 31315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:49171
I20260812 05:25:34.107993 31358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.095991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.24.66.126:45023)
0812 05:25:34.098117 (+ 2126us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.098119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.098127 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:34.098148 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.098149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.098158 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.098211 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:34.101156 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.101906 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.102789 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.102997 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.103101 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.104920 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:34.104922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:34.104923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:34.104930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:34.106632 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.107115 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.107116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.107116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.107180 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.107379 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.107380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.107380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.107515 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:34.107635 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.107774 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.107861 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2112,"thread_start_us":53,"threads_started":1}
I20260812 05:25:34.108685 32638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.096383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45736 (local address 127.24.66.125:44217)
0812 05:25:34.097052 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.097054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.097433 (+ 379us) server_negotiation.cc:407] Connection header received
0812 05:25:34.097457 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.097458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.097471 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.097532 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:34.098376 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.099135 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.101292 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.101594 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.101695 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.104405 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:34.104407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:34.104408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:34.104418 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:34.106919 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.107327 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.107328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.107328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.107401 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.107618 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.107619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.107620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.107775 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:34.107882 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.108379 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.108499 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":67,"threads_started":1}
I20260812 05:25:34.108968 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.096782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39816 (local address 127.24.66.124:46257)
0812 05:25:34.098174 (+ 1392us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.098176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.101418 (+ 3242us) server_negotiation.cc:407] Connection header received
0812 05:25:34.101503 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.101504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.101515 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.101543 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:34.102378 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.103154 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.104396 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.104580 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.104668 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.106036 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:34.106038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:34.106038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:34.106046 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:34.107754 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.108136 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.108137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.108138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.108202 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.108408 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.108409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.108409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.108534 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:34.108641 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.108783 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.108871 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1376,"thread_start_us":64,"threads_started":1}
I20260812 05:25:34.110235 31315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:34.112129 31315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:34.113044 32114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: NON_VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: true } } }
I20260812 05:25:34.113173 32114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:34.115003 32114 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037: attempting to promote NON_VOTER 2ca4ea1990cb4214ae859a9978e75b65 to VOTER
I20260812 05:25:34.115198 32114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } }
I20260812 05:25:34.115829 31960 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 FOLLOWER]: Refusing update from remote peer 0ec15ce8ef3740b6abd2322926963037: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 05:25:34.116017 32160 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 LEARNER]: Refusing update from remote peer 0ec15ce8ef3740b6abd2322926963037: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 05:25:34.116135 32643 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 05:25:34.116353 32644 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 05:25:34.117050 32644 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2ca4ea1990cb4214ae859a9978e75b65 (127.24.66.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:34.117190 31960 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2ca4ea1990cb4214ae859a9978e75b65 (127.24.66.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:34.117282 32160 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2ca4ea1990cb4214ae859a9978e75b65 (127.24.66.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:34.118197 32644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:34.118290 32644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:34.118393 32192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:34.118480 32192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:25:34.118561 32644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:34.118572 32114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:34.118705 32114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:34.118649 32644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:34.119164 32654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ec15ce8ef3740b6abd2322926963037" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:34.119274 32654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:25:34.123201 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.118866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:34223 (local address 127.0.0.1:47134)
0812 05:25:34.119158 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:34.119171 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:34.119227 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:34.119606 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:34.119607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:34.119610 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:34.119662 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.119664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.120774 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.120775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.121985 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.121987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.122962 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.122963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.123027 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.123032 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:25:34.123079 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 05:25:34.123120 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"spinlock_wait_cycles":896}
I20260812 05:25:34.123507 32655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.118969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.24.66.67:34223)
0812 05:25:34.119362 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.119367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.119378 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:34.119422 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.119425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.119443 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.119528 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:34.119770 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.120688 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.122087 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.122872 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.122950 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.123101 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.123131 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.123195 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":118,"threads_started":1}
I20260812 05:25:34.123641 31909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.118962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:42989 (local address 127.0.0.1:43132)
0812 05:25:34.119141 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:34.119164 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:34.119211 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:34.119841 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:34.119842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:34.119843 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:34.119874 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.119876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.121335 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.121336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.122291 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.122294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.123405 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.123406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.123479 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.123482 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:34.123514 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 05:25:34.123560 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":131,"threads_started":1}
I20260812 05:25:34.124099 32658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.118995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:38509 (local address 127.0.0.1:44078)
0812 05:25:34.119938 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:34.119950 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:34.120014 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:34.120399 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:34.120400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:34.120402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:34.120443 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.120445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.121768 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.121769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.122654 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.122656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.123874 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.123875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.123938 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.123943 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:25:34.123993 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 05:25:34.124035 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"thread_start_us":118,"threads_started":1}
I20260812 05:25:34.124145 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.119413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43132 (local address 127.24.66.65:42989)
0812 05:25:34.119592 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.119597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.119607 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:34.119717 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.119720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.119738 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.119819 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:34.119932 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.121229 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.122442 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.123229 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.123837 (+ 608us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.123992 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.124023 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.124068 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260812 05:25:34.124212 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.119269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.24.66.66:38509)
0812 05:25:34.120185 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.120189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.120199 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:34.120237 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.120239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.120254 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.120328 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:34.120513 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.121683 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.122747 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.123483 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.123577 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.123747 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.124024 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.124069 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":97,"threads_started":1}
I20260812 05:25:34.124523 32554 tablet_service.cc:1467] Processing CreateTablet for tablet ba6016ea97984638b9543b0b6a00f1b8 (DEFAULT_TABLE table=test-table [id=1e7c5952beab4a9093b8a4c43c1e4192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:34.125001 32554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6016ea97984638b9543b0b6a00f1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:34.124925 32269 tablet_service.cc:1467] Processing CreateTablet for tablet ba6016ea97984638b9543b0b6a00f1b8 (DEFAULT_TABLE table=test-table [id=1e7c5952beab4a9093b8a4c43c1e4192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:34.125314 32269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6016ea97984638b9543b0b6a00f1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:34.130779 32661 tablet_bootstrap.cc:492] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: Bootstrap starting.
I20260812 05:25:34.131320 32413 tablet_service.cc:1467] Processing CreateTablet for tablet ba6016ea97984638b9543b0b6a00f1b8 (DEFAULT_TABLE table=test-table [id=1e7c5952beab4a9093b8a4c43c1e4192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:34.131901 32413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba6016ea97984638b9543b0b6a00f1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:34.132130 32661 tablet_bootstrap.cc:654] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:34.132807 32661 log.cc:824] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:34.134068 32661 tablet_bootstrap.cc:492] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: No bootstrap required, opened a new log
I20260812 05:25:34.134164 32661 ts_tablet_manager.cc:1397] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:25:34.135546 32662 tablet_bootstrap.cc:492] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1: Bootstrap starting.
I20260812 05:25:34.136576 32662 tablet_bootstrap.cc:654] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:34.137162 32662 log.cc:824] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:34.138244 32662 tablet_bootstrap.cc:492] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1: No bootstrap required, opened a new log
I20260812 05:25:34.138674 32662 ts_tablet_manager.cc:1397] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:25:34.139493 32661 raft_consensus.cc:348] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:34.139680 32661 raft_consensus.cc:374] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:34.139708 32661 raft_consensus.cc:729] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 044a7521d21e4c0b93be61cd66d6bea3, State: Initialized, Role: FOLLOWER
I20260812 05:25:34.139885 32661 consensus_queue.cc:260] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:34.140414 32661 ts_tablet_manager.cc:1428] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260812 05:25:34.140522 32341 heartbeater.cc:503] Master 127.24.66.126:45023 was elected leader, sending a full tablet report...
I20260812 05:25:34.142643 32665 tablet_bootstrap.cc:492] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Bootstrap starting.
I20260812 05:25:34.143627 32665 tablet_bootstrap.cc:654] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:34.143518 32662 raft_consensus.cc:348] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:34.143702 32662 raft_consensus.cc:374] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:34.143767 32662 raft_consensus.cc:729] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86dc52beff2747f9bbe8f6041061d1a1, State: Initialized, Role: FOLLOWER
I20260812 05:25:34.143901 32662 consensus_queue.cc:260] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:34.144253 32662 ts_tablet_manager.cc:1428] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 05:25:34.144378 32665 log.cc:824] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:34.144593 32625 heartbeater.cc:503] Master 127.24.66.126:45023 was elected leader, sending a full tablet report...
I20260812 05:25:34.145740 32665 tablet_bootstrap.cc:492] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: No bootstrap required, opened a new log
I20260812 05:25:34.145807 32665 ts_tablet_manager.cc:1397] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:25:34.148452 32665 raft_consensus.cc:348] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:34.148602 32665 raft_consensus.cc:374] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:34.148627 32665 raft_consensus.cc:729] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e191f23bc0c4e04aa4ec9cc21fbf4f4, State: Initialized, Role: FOLLOWER
I20260812 05:25:34.148752 32665 consensus_queue.cc:260] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:34.149032 32665 ts_tablet_manager.cc:1428] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260812 05:25:34.149178 32483 heartbeater.cc:503] Master 127.24.66.126:45023 was elected leader, sending a full tablet report...
W20260812 05:25:34.174945 32485 tablet.cc:2369] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:25:34.239540 32666 raft_consensus.cc:482] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:34.239741 32666 raft_consensus.cc:504] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:34.240195 32666 leader_election.cc:288] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86dc52beff2747f9bbe8f6041061d1a1 (127.24.66.67:34223), 7e191f23bc0c4e04aa4ec9cc21fbf4f4 (127.24.66.66:38509)
I20260812 05:25:34.243472 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.240403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:38509 (local address 127.24.66.65:44393)
0812 05:25:34.240452 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:34.240464 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:34.240511 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:34.240642 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:34.240643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:34.240645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:34.240730 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.240732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.241465 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.241466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.242521 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.242524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.243151 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.243152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.243252 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.243256 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:34.243325 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:25:34.243370 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 05:25:34.243508 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.240450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:44393 (local address 127.24.66.66:38509)
0812 05:25:34.240500 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.240502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.240513 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:34.240570 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.240570 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.240580 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.240639 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:34.240819 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.241384 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.242605 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.243073 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.243130 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.243137 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.243366 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.243403 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:25:34.243937 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.240385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:34223 (local address 127.24.66.65:55927)
0812 05:25:34.240452 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:34.240463 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:34.240509 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:34.240759 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:34.240760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:34.240761 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:34.240846 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.240848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.241821 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.241822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.242740 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:34.242743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.243490 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.243491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:34.243686 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.243691 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:25:34.243758 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:25:34.243805 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 05:25:34.243907 32434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6016ea97984638b9543b0b6a00f1b8" candidate_uuid: "044a7521d21e4c0b93be61cd66d6bea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" is_pre_election: true
I20260812 05:25:34.244084 32434 raft_consensus.cc:2454] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 044a7521d21e4c0b93be61cd66d6bea3 in term 0.
I20260812 05:25:34.244248 32655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.240553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:55927 (local address 127.24.66.67:34223)
0812 05:25:34.240638 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.240640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.240649 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:34.240678 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.240679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.240689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.240739 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:34.240919 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.241681 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.242836 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.243398 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.243471 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.243479 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.244104 (+ 625us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.244184 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:25:34.244371 32223 leader_election.cc:302] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 044a7521d21e4c0b93be61cd66d6bea3, 7e191f23bc0c4e04aa4ec9cc21fbf4f4; no voters:
I20260812 05:25:34.244524 32666 raft_consensus.cc:2781] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:34.244573 32666 raft_consensus.cc:482] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:34.244601 32666 raft_consensus.cc:3037] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:34.244663 32576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6016ea97984638b9543b0b6a00f1b8" candidate_uuid: "044a7521d21e4c0b93be61cd66d6bea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86dc52beff2747f9bbe8f6041061d1a1" is_pre_election: true
I20260812 05:25:34.244846 32576 raft_consensus.cc:2454] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 044a7521d21e4c0b93be61cd66d6bea3 in term 0.
I20260812 05:25:34.245537 32666 raft_consensus.cc:504] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:34.245685 32666 leader_election.cc:288] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [CANDIDATE]: Term 1 election: Requested vote from peers 86dc52beff2747f9bbe8f6041061d1a1 (127.24.66.67:34223), 7e191f23bc0c4e04aa4ec9cc21fbf4f4 (127.24.66.66:38509)
I20260812 05:25:34.245812 32576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6016ea97984638b9543b0b6a00f1b8" candidate_uuid: "044a7521d21e4c0b93be61cd66d6bea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86dc52beff2747f9bbe8f6041061d1a1"
I20260812 05:25:34.245852 32434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba6016ea97984638b9543b0b6a00f1b8" candidate_uuid: "044a7521d21e4c0b93be61cd66d6bea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4"
I20260812 05:25:34.245918 32576 raft_consensus.cc:3037] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:34.245924 32434 raft_consensus.cc:3037] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:34.246909 32576 raft_consensus.cc:2454] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 044a7521d21e4c0b93be61cd66d6bea3 in term 1.
I20260812 05:25:34.246925 32434 raft_consensus.cc:2454] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 044a7521d21e4c0b93be61cd66d6bea3 in term 1.
I20260812 05:25:34.247200 32221 leader_election.cc:302] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 044a7521d21e4c0b93be61cd66d6bea3, 86dc52beff2747f9bbe8f6041061d1a1; no voters:
I20260812 05:25:34.247316 32666 raft_consensus.cc:2781] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:34.247469 32666 raft_consensus.cc:686] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 1 LEADER]: Becoming Leader. State: Replica: 044a7521d21e4c0b93be61cd66d6bea3, State: Running, Role: LEADER
I20260812 05:25:34.247581 32666 consensus_queue.cc:237] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:34.248549 31315 catalog_manager.cc:5495] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 044a7521d21e4c0b93be61cd66d6bea3 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "044a7521d21e4c0b93be61cd66d6bea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } health_report { overall_health: HEALTHY } } }
W20260812 05:25:34.259690 32343 tablet.cc:2369] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:25:34.269454 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.266769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43142 (local address 127.24.66.65:42989)
0812 05:25:34.266822 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.266824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.266832 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:34.266937 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.266938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.266956 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.267019 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:34.267156 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.267618 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.268578 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.268797 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.268858 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.268997 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:34.269101 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:34.269163 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.269277 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.269328 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:25:34.271651 32434 raft_consensus.cc:1261] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 1 FOLLOWER]: Refusing update from remote peer 044a7521d21e4c0b93be61cd66d6bea3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:34.271662 32576 raft_consensus.cc:1261] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1 [term 1 FOLLOWER]: Refusing update from remote peer 044a7521d21e4c0b93be61cd66d6bea3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:34.272061 32666 consensus_queue.cc:1035] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:34.272223 32670 consensus_queue.cc:1035] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:34.276916 32674 mvcc.cc:205] Tried to move back new op lower bound from 7317554521173598208 to 7317554521078734848. Current Snapshot: MvccSnapshot[applied={T|T < 7317554521173598208}]
I20260812 05:25:34.278103 32675 mvcc.cc:205] Tried to move back new op lower bound from 7317554521173598208 to 7317554521078734848. Current Snapshot: MvccSnapshot[applied={T|T < 7317554521173598208}]
I20260812 05:25:34.279377 31314 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:34.279486 31314 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:34.280980 31314 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ec15ce8ef3740b6abd2322926963037: Sending DeleteTablet for 3 replicas of tablet ba6016ea97984638b9543b0b6a00f1b8
I20260812 05:25:34.281383 32413 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6016ea97984638b9543b0b6a00f1b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:34 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:44078
I20260812 05:25:34.281414 32554 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6016ea97984638b9543b0b6a00f1b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:34 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:47134
I20260812 05:25:34.281617 32269 tablet_service.cc:1514] Processing DeleteTablet for tablet ba6016ea97984638b9543b0b6a00f1b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:34 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:43132
I20260812 05:25:34.282804 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 31280
I20260812 05:25:34.286478 32686 tablet_replica.cc:330] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: stopping tablet replica
I20260812 05:25:34.286677 32686 raft_consensus.cc:2229] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:34.286828 32686 raft_consensus.cc:2258] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:34.287196 32684 tablet_replica.cc:330] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1: stopping tablet replica
I20260812 05:25:34.287325 32684 raft_consensus.cc:2229] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:34.287426 32684 raft_consensus.cc:2258] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:34.287799 32686 ts_tablet_manager.cc:1905] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:34.289816 32686 ts_tablet_manager.cc:1918] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:34.289992 32686 log.cc:1196] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-0/wal/wals/ba6016ea97984638b9543b0b6a00f1b8
I20260812 05:25:34.290486 32686 ts_tablet_manager.cc:1939] T ba6016ea97984638b9543b0b6a00f1b8 P 044a7521d21e4c0b93be61cd66d6bea3: Deleting consensus metadata
I20260812 05:25:34.292575 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.282418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39822 (local address 127.24.66.124:46257)
0812 05:25:34.282528 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.282530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.282539 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:34.282562 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.282563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.282574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.282642 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:34.284030 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.284906 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.286150 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.286493 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.286547 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.288861 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:34.288863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:34.288863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:34.288873 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:34.290905 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.291410 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.291412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.291413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.291493 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.291748 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.291749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.291749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.291937 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:34.292057 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.292287 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.292403 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260812 05:25:34.294229 32221 connection.cc:535] server connection from 127.0.0.1:43132 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 05:25:34.295009 32684 ts_tablet_manager.cc:1905] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:34.297008 32684 ts_tablet_manager.cc:1918] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:34.297116 32684 log.cc:1196] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/ba6016ea97984638b9543b0b6a00f1b8
I20260812 05:25:34.297467 32684 ts_tablet_manager.cc:1939] T ba6016ea97984638b9543b0b6a00f1b8 P 86dc52beff2747f9bbe8f6041061d1a1: Deleting consensus metadata
I20260812 05:25:34.298033 32685 tablet_replica.cc:330] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: stopping tablet replica
W20260812 05:25:34.298002 32507 connection.cc:441] server connection from 127.0.0.1:47134 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47134 (request call id 1) could send its response
I20260812 05:25:34.298174 32685 raft_consensus.cc:2229] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:34.298331 32685 raft_consensus.cc:2258] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:34.308156 32638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.296578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.24.66.125:44217)
0812 05:25:34.296647 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.296649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.297219 (+ 570us) server_negotiation.cc:407] Connection header received
0812 05:25:34.297371 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.297373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.297395 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.297469 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:34.299432 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.300359 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.301691 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.302080 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.302162 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.304232 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:34.304234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:34.304235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:34.304245 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:34.306641 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.307105 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.307107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.307107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.307171 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.307409 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.307410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.307410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.307566 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:34.307666 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.307914 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.308028 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:25:34.308857 32346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:34.298872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39828 (local address 127.24.66.124:46257)
0812 05:25:34.299332 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:34.299334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:34.299346 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:25:34.299376 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:34.299378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:34.299391 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:34.299454 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:34.300746 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.301600 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.302577 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:34.302925 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:34.302973 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:34.305059 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:34.305060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:34.305060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:34.305070 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:34.307463 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.307896 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.307897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.307897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.307958 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:34.308198 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:34.308199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:34.308200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:34.308362 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:34.308435 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:34.308664 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 05:25:34.308745 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423}
I20260812 05:25:34.310904 32685 ts_tablet_manager.cc:1905] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:34.312863 32685 ts_tablet_manager.cc:1918] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:34.312969 32685 log.cc:1196] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/ba6016ea97984638b9543b0b6a00f1b8
I20260812 05:25:34.313351 32685 ts_tablet_manager.cc:1939] T ba6016ea97984638b9543b0b6a00f1b8 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Deleting consensus metadata
W20260812 05:25:34.313894 32365 connection.cc:441] server connection from 127.0.0.1:44078 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44078 (request call id 1) could send its response
I20260812 05:25:35.277959 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.277565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.67:55121)
0812 05:25:35.277751 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.277894 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
I20260812 05:25:35.278105 32695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.277796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.66:53571)
0812 05:25:35.277967 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.278049 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":64,"threads_started":1}
W20260812 05:25:35.278326 32483 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:45023 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
I20260812 05:25:35.278546 32696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.278125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.65:45257)
0812 05:25:35.278425 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.278501 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":55,"threads_started":1}
W20260812 05:25:35.278637 32625 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:45023 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
W20260812 05:25:35.278959 32341 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:45023 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
I20260812 05:25:35.782166 32697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec15ce8ef3740b6abd2322926963037)
I20260812 05:25:35.782403 32697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } }
I20260812 05:25:35.782759 32697 leader_election.cc:288] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec15ce8ef3740b6abd2322926963037 (127.24.66.126:45023), 63fb1e36d61646058cb8282b12a09e8e (127.24.66.125:44217)
I20260812 05:25:35.782785 32698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0ec15ce8ef3740b6abd2322926963037)
I20260812 05:25:35.782939 32698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } }
I20260812 05:25:35.783156 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.782917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43744)
0812 05:25:35.783036 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.783111 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
W20260812 05:25:35.783386 32136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
I20260812 05:25:35.783551 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.783302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43746)
0812 05:25:35.783401 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.783505 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":48,"threads_started":1}
W20260812 05:25:35.783771 31936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
I20260812 05:25:35.783806 32698 leader_election.cc:288] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec15ce8ef3740b6abd2322926963037 (127.24.66.126:45023), 2ca4ea1990cb4214ae859a9978e75b65 (127.24.66.124:46257)
I20260812 05:25:35.783968 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.783711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43758)
0812 05:25:35.783878 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.783920 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
W20260812 05:25:35.784065 32136 leader_election.cc:334] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ec15ce8ef3740b6abd2322926963037 (127.24.66.126:45023): Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
I20260812 05:25:35.784454 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.784311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43770)
0812 05:25:35.784377 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.784416 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 05:25:35.784549 31936 leader_election.cc:334] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0ec15ce8ef3740b6abd2322926963037 (127.24.66.126:45023): Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
I20260812 05:25:35.786916 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.783288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44217 (local address 127.0.0.1:45766)
0812 05:25:35.783346 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.783357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:35.783433 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:35.783647 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:35.783648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:35.783652 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:35.783720 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.783722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.784776 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.784777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.785879 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.785882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.786721 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.786722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.786777 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.786782 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:25:35.786831 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 05:25:35.786867 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":2}
I20260812 05:25:35.786955 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.783159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45766 (local address 127.24.66.125:44217)
0812 05:25:35.783302 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:35.783305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:35.783494 (+ 189us) server_negotiation.cc:407] Connection header received
0812 05:25:35.783522 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:35.783523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:35.783534 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:35.783849 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:35.783896 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.784671 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.785991 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.786632 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.786722 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.786735 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:35.786848 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 05:25:35.786889 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260812 05:25:35.787185 31960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ca4ea1990cb4214ae859a9978e75b65" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "63fb1e36d61646058cb8282b12a09e8e" is_pre_election: true
I20260812 05:25:35.787302 31960 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ca4ea1990cb4214ae859a9978e75b65 in term 1.
I20260812 05:25:35.787405 32704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.783712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39842 (local address 127.24.66.124:46257)
0812 05:25:35.783917 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:35.783919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:35.784102 (+ 183us) server_negotiation.cc:407] Connection header received
0812 05:25:35.784128 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:35.784129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:35.784141 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:35.784212 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:35.784499 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.785141 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.786411 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.787014 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.787086 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.787097 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:35.787298 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 05:25:35.787339 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":67,"threads_started":1}
I20260812 05:25:35.787446 32703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.783507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:46257 (local address 127.0.0.1:39842)
0812 05:25:35.783961 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.783972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:35.784039 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:35.784319 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:35.784320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:35.784323 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:35.784404 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.784407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.785246 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.785247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.786291 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.786294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.787139 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.787140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.787212 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.787216 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:35.787278 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 05:25:35.787321 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":44,"threads_started":1}
I20260812 05:25:35.787588 32136 leader_election.cc:302] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ca4ea1990cb4214ae859a9978e75b65, 63fb1e36d61646058cb8282b12a09e8e; no voters: 0ec15ce8ef3740b6abd2322926963037
I20260812 05:25:35.787705 32697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 05:25:35.787698 32160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63fb1e36d61646058cb8282b12a09e8e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2ca4ea1990cb4214ae859a9978e75b65" is_pre_election: true
I20260812 05:25:35.787819 32697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0ec15ce8ef3740b6abd2322926963037)
I20260812 05:25:35.787851 32697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:25:35.788663 32697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } }
I20260812 05:25:35.788825 32697 leader_election.cc:288] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [CANDIDATE]: Term 2 election: Requested vote from peers 0ec15ce8ef3740b6abd2322926963037 (127.24.66.126:45023), 63fb1e36d61646058cb8282b12a09e8e (127.24.66.125:44217)
I20260812 05:25:35.788982 32160 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 63fb1e36d61646058cb8282b12a09e8e in current term 2: Already voted for candidate 2ca4ea1990cb4214ae859a9978e75b65 in this term.
I20260812 05:25:35.789027 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.788910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43792)
0812 05:25:35.788964 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.789000 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 05:25:35.789062 31960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2ca4ea1990cb4214ae859a9978e75b65" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "63fb1e36d61646058cb8282b12a09e8e"
I20260812 05:25:35.789149 31960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:25:35.789438 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.789328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43796)
0812 05:25:35.789391 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.789414 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 05:25:35.789546 31934 leader_election.cc:302] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63fb1e36d61646058cb8282b12a09e8e; no voters: 0ec15ce8ef3740b6abd2322926963037, 2ca4ea1990cb4214ae859a9978e75b65
W20260812 05:25:35.789569 32136 leader_election.cc:334] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0ec15ce8ef3740b6abd2322926963037 (127.24.66.126:45023): Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
I20260812 05:25:35.789927 31960 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ca4ea1990cb4214ae859a9978e75b65 in term 2.
I20260812 05:25:35.790072 32698 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:25:35.790128 32136 leader_election.cc:302] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ca4ea1990cb4214ae859a9978e75b65, 63fb1e36d61646058cb8282b12a09e8e; no voters: 0ec15ce8ef3740b6abd2322926963037
I20260812 05:25:35.790220 32697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 05:25:35.790416 32697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [term 2 LEADER]: Becoming Leader. State: Replica: 2ca4ea1990cb4214ae859a9978e75b65, State: Running, Role: LEADER
I20260812 05:25:35.790489 32697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } }
I20260812 05:25:35.790884 32707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ca4ea1990cb4214ae859a9978e75b65. Latest consensus state: current_term: 2 leader_uuid: "2ca4ea1990cb4214ae859a9978e75b65" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:35.790990 32707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:35.791191 32711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:35.791723 32711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:35.791829 32711 catalog_manager.cc:1259] Loaded cluster ID: d3b38d4bde584a1d9c334521c2e37ce6
I20260812 05:25:35.791863 32711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:35.792030 32711 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:35.792142 32711 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: Loaded TSK: 0
I20260812 05:25:35.792310 32711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:25:35.848438 32149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:35.850107 32149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:35.851354 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.851223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43818)
0812 05:25:35.851281 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.851320 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 05:25:35.851366 31960 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [term 2 FOLLOWER]: Refusing update from remote peer 2ca4ea1990cb4214ae859a9978e75b65: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 05:25:35.851630 32697 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 05:25:35.851799 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.851685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.0.0.1:43822)
0812 05:25:35.851744 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.851776 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 05:25:35.851984 32136 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 -> Peer 0ec15ce8ef3740b6abd2322926963037 (127.24.66.126:45023): Couldn't send request to peer 0ec15ce8ef3740b6abd2322926963037. Status: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 05:25:35.852507 32698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ca4ea1990cb4214ae859a9978e75b65. Latest consensus state: current_term: 2 leader_uuid: "2ca4ea1990cb4214ae859a9978e75b65" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:35.852614 32698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:25:35.853087 32697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "2ca4ea1990cb4214ae859a9978e75b65" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:35.853184 32697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65 [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:35.853199 32698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "2ca4ea1990cb4214ae859a9978e75b65" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "0ec15ce8ef3740b6abd2322926963037" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45023 } } peers { permanent_uuid: "63fb1e36d61646058cb8282b12a09e8e" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44217 } attrs { promote: false } } peers { permanent_uuid: "2ca4ea1990cb4214ae859a9978e75b65" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 46257 } attrs { promote: false } } }
I20260812 05:25:35.853295 32698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63fb1e36d61646058cb8282b12a09e8e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:25:35.853368 32713 mvcc.cc:205] Tried to move back new op lower bound from 7317554527644778496 to 7317554527398309888. Current Snapshot: MvccSnapshot[applied={T|T < 7317554527644778496}]
I20260812 05:25:35.858210 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.855212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:42989 (local address 127.0.0.1:43144)
0812 05:25:35.855475 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.855484 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:35.855522 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:35.855731 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:35.855732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:35.855733 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:35.855754 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.855755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.856579 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.856580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.857348 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.857350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.857994 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.857995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.858068 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.858074 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:25:35.858106 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 05:25:35.858145 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":28,"thread_start_us":92,"threads_started":1}
I20260812 05:25:35.858556 32719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.855287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43144 (local address 127.24.66.65:42989)
0812 05:25:35.855615 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:35.855617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:35.855624 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:25:35.855650 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:35.855651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:35.855661 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:35.855731 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:35.855893 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.856492 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.857438 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.857900 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.858388 (+ 488us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.858407 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:35.858427 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 05:25:35.858474 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":64,"threads_started":1}
I20260812 05:25:35.858842 32269 tablet_service.cc:1467] Processing CreateTablet for tablet bd1d7c44ebf546748ef0cb47be952998 (DEFAULT_TABLE table=test-table [id=2b4feea6a99b4e4184618536f9a067f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:35.858999 32269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd1d7c44ebf546748ef0cb47be952998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:35.859040 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.855152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.24.66.66:38509)
0812 05:25:35.855428 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:35.855430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:35.855441 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:35.855476 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:35.855476 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:35.855490 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:35.855568 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:35.855660 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.856544 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.857927 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.858552 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.858621 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.858633 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:35.858923 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 05:25:35.858966 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":72,"threads_started":1}
I20260812 05:25:35.859040 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.855088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:38509 (local address 127.0.0.1:44080)
0812 05:25:35.855357 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.855370 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:35.855423 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:35.855568 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:35.855568 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:35.855570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:35.855616 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.855618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.856640 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.856641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.857810 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.857820 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.858779 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.858780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.858856 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.858862 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:25:35.858916 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 05:25:35.858965 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242}
I20260812 05:25:35.859153 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.855117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:34223 (local address 127.0.0.1:47144)
0812 05:25:35.855352 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.855364 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:35.855411 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:35.855944 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:35.855945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:35.855947 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:35.855987 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.855989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.856931 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.856932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.857942 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.857944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.858858 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.858859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.858923 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.858933 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 05:25:35.859017 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 05:25:35.859063 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260812 05:25:35.859305 32413 tablet_service.cc:1467] Processing CreateTablet for tablet bd1d7c44ebf546748ef0cb47be952998 (DEFAULT_TABLE table=test-table [id=2b4feea6a99b4e4184618536f9a067f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:35.859369 32720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.855289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47144 (local address 127.24.66.67:34223)
0812 05:25:35.855807 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:35.855809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:35.855817 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:35.855842 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:35.855843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:35.855853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:35.856052 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:35.856107 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.856843 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.858032 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.858743 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.858839 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.858852 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:35.859240 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 05:25:35.859292 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":76,"threads_started":1}
I20260812 05:25:35.859414 32413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd1d7c44ebf546748ef0cb47be952998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:35.859614 32554 tablet_service.cc:1467] Processing CreateTablet for tablet bd1d7c44ebf546748ef0cb47be952998 (DEFAULT_TABLE table=test-table [id=2b4feea6a99b4e4184618536f9a067f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:35.859725 32554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd1d7c44ebf546748ef0cb47be952998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:35.860829 32721 tablet_bootstrap.cc:492] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3: Bootstrap starting.
I20260812 05:25:35.861032 32722 tablet_bootstrap.cc:492] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Bootstrap starting.
I20260812 05:25:35.861349 32721 tablet_bootstrap.cc:654] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:35.861483 32722 tablet_bootstrap.cc:654] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:35.861793 32723 tablet_bootstrap.cc:492] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1: Bootstrap starting.
I20260812 05:25:35.861871 32721 tablet_bootstrap.cc:492] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3: No bootstrap required, opened a new log
I20260812 05:25:35.861945 32721 ts_tablet_manager.cc:1397] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:35.862068 32721 raft_consensus.cc:348] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:35.862135 32721 raft_consensus.cc:374] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:35.862149 32721 raft_consensus.cc:729] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 044a7521d21e4c0b93be61cd66d6bea3, State: Initialized, Role: FOLLOWER
I20260812 05:25:35.862208 32723 tablet_bootstrap.cc:654] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:35.862205 32721 consensus_queue.cc:260] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:35.862372 32721 ts_tablet_manager.cc:1428] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:25:35.862825 32728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.862583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.65:44275)
0812 05:25:35.862739 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.862784 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260812 05:25:35.863070 32722 tablet_bootstrap.cc:492] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: No bootstrap required, opened a new log
I20260812 05:25:35.863247 32722 ts_tablet_manager.cc:1397] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:25:35.863235 32340 heartbeater.cc:503] Master 127.24.66.124:46257 was elected leader, sending a full tablet report...
I20260812 05:25:35.863237 32728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.863047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.65:39221)
0812 05:25:35.863180 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.863209 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260812 05:25:35.863394 32722 raft_consensus.cc:348] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:35.863471 32722 raft_consensus.cc:374] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:35.863492 32722 raft_consensus.cc:729] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e191f23bc0c4e04aa4ec9cc21fbf4f4, State: Initialized, Role: FOLLOWER
I20260812 05:25:35.863555 32722 consensus_queue.cc:260] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:35.863749 32722 ts_tablet_manager.cc:1428] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:35.863842 32723 tablet_bootstrap.cc:492] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1: No bootstrap required, opened a new log
I20260812 05:25:35.863897 32723 ts_tablet_manager.cc:1397] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:25:35.864028 32723 raft_consensus.cc:348] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:35.864094 32723 raft_consensus.cc:374] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:35.864118 32723 raft_consensus.cc:729] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86dc52beff2747f9bbe8f6041061d1a1, State: Initialized, Role: FOLLOWER
I20260812 05:25:35.864190 32723 consensus_queue.cc:260] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:35.864284 32482 heartbeater.cc:503] Master 127.24.66.124:46257 was elected leader, sending a full tablet report...
I20260812 05:25:35.864357 32723 ts_tablet_manager.cc:1428] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:35.864456 32732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.864049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.66:53999)
0812 05:25:35.864368 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.864408 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":104,"threads_started":1}
I20260812 05:25:35.864800 32624 heartbeater.cc:503] Master 127.24.66.124:46257 was elected leader, sending a full tablet report...
I20260812 05:25:35.864861 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.864606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.67:58887)
0812 05:25:35.864783 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.864821 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260812 05:25:35.865211 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.865092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.67:45873)
0812 05:25:35.865131 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.865157 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 05:25:35.971885 32733 raft_consensus.cc:482] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:35.971987 32733 raft_consensus.cc:504] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:35.972406 32733 leader_election.cc:288] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e191f23bc0c4e04aa4ec9cc21fbf4f4 (127.24.66.66:38509), 044a7521d21e4c0b93be61cd66d6bea3 (127.24.66.65:42989)
I20260812 05:25:35.975525 32719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.972735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:51491 (local address 127.24.66.65:42989)
0812 05:25:35.972796 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:35.972799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:35.972867 (+ 68us) server_negotiation.cc:407] Connection header received
0812 05:25:35.972934 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:35.972936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:35.972946 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:35.973026 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:35.973193 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.973724 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.974801 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.975169 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.975259 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.975270 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:35.975416 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 05:25:35.975454 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:25:35.975540 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.972654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:42989 (local address 127.24.66.67:51491)
0812 05:25:35.972756 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.972769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:35.972872 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:35.973030 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:35.973031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:35.973034 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:35.973120 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.973122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.973829 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.973830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.974705 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.974708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.975259 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.975260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.975336 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.975339 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:35.975409 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 05:25:35.975448 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 05:25:35.975935 32735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.972654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:38509 (local address 127.24.66.67:57705)
0812 05:25:35.972964 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.972982 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:35.973072 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:35.973246 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:35.973247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:35.973249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:35.973331 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.973333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.974087 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.974088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.974995 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:35.974998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.975633 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.975634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:35.975722 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.975726 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:35.975788 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 05:25:35.975826 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":109,"thread_start_us":147,"threads_started":1}
I20260812 05:25:35.975958 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.972820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:57705 (local address 127.24.66.66:38509)
0812 05:25:35.972876 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:35.972878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:35.973061 (+ 183us) server_negotiation.cc:407] Connection header received
0812 05:25:35.973151 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:35.973152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:35.973163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:35.973243 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:35.973416 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.973984 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.975103 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:35.975525 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:35.975622 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:35.975639 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:35.975835 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 05:25:35.975883 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 05:25:35.975916 32290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd1d7c44ebf546748ef0cb47be952998" candidate_uuid: "86dc52beff2747f9bbe8f6041061d1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "044a7521d21e4c0b93be61cd66d6bea3" is_pre_election: true
I20260812 05:25:35.976059 32290 raft_consensus.cc:2454] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86dc52beff2747f9bbe8f6041061d1a1 in term 0.
I20260812 05:25:35.976316 32506 leader_election.cc:302] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 044a7521d21e4c0b93be61cd66d6bea3, 86dc52beff2747f9bbe8f6041061d1a1; no voters:
I20260812 05:25:35.976492 32733 raft_consensus.cc:2781] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:35.976562 32733 raft_consensus.cc:482] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:35.976585 32733 raft_consensus.cc:3037] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:35.976668 32434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd1d7c44ebf546748ef0cb47be952998" candidate_uuid: "86dc52beff2747f9bbe8f6041061d1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" is_pre_election: true
I20260812 05:25:35.976768 32434 raft_consensus.cc:2454] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86dc52beff2747f9bbe8f6041061d1a1 in term 0.
I20260812 05:25:35.977404 32733 raft_consensus.cc:504] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:35.977568 32733 leader_election.cc:288] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [CANDIDATE]: Term 1 election: Requested vote from peers 7e191f23bc0c4e04aa4ec9cc21fbf4f4 (127.24.66.66:38509), 044a7521d21e4c0b93be61cd66d6bea3 (127.24.66.65:42989)
I20260812 05:25:35.977731 32434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd1d7c44ebf546748ef0cb47be952998" candidate_uuid: "86dc52beff2747f9bbe8f6041061d1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4"
I20260812 05:25:35.977771 32290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd1d7c44ebf546748ef0cb47be952998" candidate_uuid: "86dc52beff2747f9bbe8f6041061d1a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "044a7521d21e4c0b93be61cd66d6bea3"
I20260812 05:25:35.977831 32290 raft_consensus.cc:3037] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:35.977823 32434 raft_consensus.cc:3037] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:35.978531 32434 raft_consensus.cc:2454] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86dc52beff2747f9bbe8f6041061d1a1 in term 1.
I20260812 05:25:35.978703 32290 raft_consensus.cc:2454] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86dc52beff2747f9bbe8f6041061d1a1 in term 1.
I20260812 05:25:35.978785 32508 leader_election.cc:302] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e191f23bc0c4e04aa4ec9cc21fbf4f4, 86dc52beff2747f9bbe8f6041061d1a1; no voters:
I20260812 05:25:35.978883 32733 raft_consensus.cc:2781] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:35.979063 32733 raft_consensus.cc:686] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 1 LEADER]: Becoming Leader. State: Replica: 86dc52beff2747f9bbe8f6041061d1a1, State: Running, Role: LEADER
I20260812 05:25:35.979213 32733 consensus_queue.cc:237] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } } peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } }
I20260812 05:25:35.979384 32735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:35.979279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.67:41107)
0812 05:25:35.979320 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:35.979357 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 05:25:35.980118 32142 catalog_manager.cc:5495] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86dc52beff2747f9bbe8f6041061d1a1 (127.24.66.67). New cstate: current_term: 1 leader_uuid: "86dc52beff2747f9bbe8f6041061d1a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86dc52beff2747f9bbe8f6041061d1a1" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 } health_report { overall_health: UNKNOWN } } }
I20260812 05:25:36.005710 32720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:36.002732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.24.66.67:34223)
0812 05:25:36.002796 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:36.002798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:36.002830 (+ 32us) server_negotiation.cc:407] Connection header received
0812 05:25:36.002941 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:36.002942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:36.002962 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:36.003028 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:36.003141 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:36.003621 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:36.004787 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:36.005100 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:36.005181 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:36.005377 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:36.005484 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:36.005544 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:36.005580 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 05:25:36.005602 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:25:36.006978 32434 raft_consensus.cc:1261] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 1 FOLLOWER]: Refusing update from remote peer 86dc52beff2747f9bbe8f6041061d1a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:36.007066 32290 raft_consensus.cc:1261] T bd1d7c44ebf546748ef0cb47be952998 P 044a7521d21e4c0b93be61cd66d6bea3 [term 1 FOLLOWER]: Refusing update from remote peer 86dc52beff2747f9bbe8f6041061d1a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:36.007282 32736 consensus_queue.cc:1035] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e191f23bc0c4e04aa4ec9cc21fbf4f4" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:36.007401 32733 consensus_queue.cc:1035] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "044a7521d21e4c0b93be61cd66d6bea3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:36.008529 32728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:36.008396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.65:56503)
0812 05:25:36.008440 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:36.008484 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 05:25:36.008745 32732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:36.008543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.66:51215)
0812 05:25:36.008673 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:36.008707 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260812 05:25:36.009454 32735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:36.009357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.67:56783)
0812 05:25:36.009392 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:36.009421 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 05:25:36.011296 32732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:36.011071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.66:39961)
0812 05:25:36.011230 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:36.011263 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260812 05:25:36.011412 32142 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:36.011511 32142 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:36.013173 32735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:36.012842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.67:32835)
0812 05:25:36.013107 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:36.013139 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247}
I20260812 05:25:36.013224 32728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:36.013123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45023 (local address 127.24.66.65:59431)
0812 05:25:36.013158 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:36.013188 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:45023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 05:25:36.013216 32142 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ca4ea1990cb4214ae859a9978e75b65: Sending DeleteTablet for 3 replicas of tablet bd1d7c44ebf546748ef0cb47be952998
I20260812 05:25:36.013482 32413 tablet_service.cc:1514] Processing DeleteTablet for tablet bd1d7c44ebf546748ef0cb47be952998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:36 UTC) from {username='oryx', principal='oryx/127.24.66.124@KRBTEST.COM'} at 127.0.0.1:44080
I20260812 05:25:36.013537 32554 tablet_service.cc:1514] Processing DeleteTablet for tablet bd1d7c44ebf546748ef0cb47be952998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:36 UTC) from {username='oryx', principal='oryx/127.24.66.124@KRBTEST.COM'} at 127.0.0.1:47144
I20260812 05:25:36.013972 32743 mvcc.cc:205] Tried to move back new op lower bound from 7317554528282222592 to 7317554528171921408. Current Snapshot: MvccSnapshot[applied={T|T < 7317554528282222592}]
I20260812 05:25:36.014061 32269 tablet_service.cc:1514] Processing DeleteTablet for tablet bd1d7c44ebf546748ef0cb47be952998 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:36 UTC) from {username='oryx', principal='oryx/127.24.66.124@KRBTEST.COM'} at 127.0.0.1:43144
I20260812 05:25:36.014171 32751 tablet_replica.cc:330] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1: stopping tablet replica
I20260812 05:25:36.014230 32751 raft_consensus.cc:2229] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:36.014308 32751 raft_consensus.cc:2258] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:36.014394 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 32109
I20260812 05:25:36.016587 32751 ts_tablet_manager.cc:1905] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:36.018131 32751 ts_tablet_manager.cc:1918] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:36.018177 32751 log.cc:1196] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/bd1d7c44ebf546748ef0cb47be952998
I20260812 05:25:36.018431 32751 ts_tablet_manager.cc:1939] T bd1d7c44ebf546748ef0cb47be952998 P 86dc52beff2747f9bbe8f6041061d1a1: Deleting consensus metadata
I20260812 05:25:36.018786 32132 catalog_manager.cc:4841] TS 86dc52beff2747f9bbe8f6041061d1a1 (127.24.66.67:34223): tablet bd1d7c44ebf546748ef0cb47be952998 (table test-table [id=2b4feea6a99b4e4184618536f9a067f5]) successfully deleted
I20260812 05:25:36.019443 32750 tablet_replica.cc:330] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: stopping tablet replica
I20260812 05:25:36.019533 32750 raft_consensus.cc:2229] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:36.019620 32750 raft_consensus.cc:2258] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:36.019979 32750 ts_tablet_manager.cc:1905] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:36.021519 32750 ts_tablet_manager.cc:1918] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:36.021603 32750 log.cc:1196] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/bd1d7c44ebf546748ef0cb47be952998
I20260812 05:25:36.021855 32750 ts_tablet_manager.cc:1939] T bd1d7c44ebf546748ef0cb47be952998 P 7e191f23bc0c4e04aa4ec9cc21fbf4f4: Deleting consensus metadata
I20260812 05:25:36.022243 32134 catalog_manager.cc:4841] TS 7e191f23bc0c4e04aa4ec9cc21fbf4f4 (127.24.66.66:38509): tablet bd1d7c44ebf546748ef0cb47be952998 (table test-table [id=2b4feea6a99b4e4184618536f9a067f5]) successfully deleted
I20260812 05:25:36.033542 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:36.033382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:42989 (local address 127.0.0.1:43152)
0812 05:25:36.033456 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:36.033496 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:42989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 05:25:36.033560 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 32349
W20260812 05:25:36.033809 32132 catalog_manager.cc:4584] TS 044a7521d21e4c0b93be61cd66d6bea3 (127.24.66.65:42989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd1d7c44ebf546748ef0cb47be952998: Network error: Client connection negotiation failed: client connection to 127.24.66.65:42989: connect: Connection refused (error 111)
I20260812 05:25:36.039901 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 32489
I20260812 05:25:36.046977 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 31806
I20260812 05:25:36.054171 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 32008
2026-08-12T05:25:36Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5100 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:36 dist-test-slave-zpfg krb5kdc[32755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:36 dist-test-slave-zpfg krb5kdc[32755](info): set up 2 sockets
Aug 12 05:25:36 dist-test-slave-zpfg krb5kdc[32755](info): commencing operation
krb5kdc: starting...
W20260812 05:25:37.250674 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:37 dist-test-slave-zpfg krb5kdc[32755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.126.keytab.
Entry for principal oryx/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.126.keytab.
I20260812 05:25:37.391602 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:38153
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.126.keytab
--principal=oryx/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:34341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:38153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:37.479444 300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:37.479722 300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:37.479750 300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:37.480942 300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:37.480983 300 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 05:25:37.480995 300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:37.481009 300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:37.481021 300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:37.481033 300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:37.482658 300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:38153
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.126.keytab
--principal=oryx/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:38153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:37.482877 300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:37.483062 300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:37.488376 305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:37.488420 308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:37.488442 300 server_base.cc:1034] running on GCE node
W20260812 05:25:37.488407 306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:37.489055 300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:37.489472 300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:37.490628 300 hybrid_clock.cc:648] HybridClock initialized: now 1786512337490607 us; error 40 us; skew 500 ppm
Aug 12 05:25:37 dist-test-slave-zpfg krb5kdc[32755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512337, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:37.493389 300 init.cc:377] Logged in from keytab as oryx/127.24.66.126@KRBTEST.COM (short username oryx)
I20260812 05:25:37.493919 300 webserver.cc:460] Webserver started at http://127.24.66.126:34545/ using document root <none> and password file <none>
I20260812 05:25:37.494175 300 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:37.494223 300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:37.494444 300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:37.495576 300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "a832ab86cff0417ba28fdaeea31d137e"
format_stamp: "Formatted at 2026-08-12 05:25:37 on dist-test-slave-zpfg"
server_key: "4e1433ffb647196a3b7ff4fcbfee8b0c"
server_key_iv: "e4bd0aa03e9e020067896501cc6db2a7"
server_key_version: "encryptionkey@0"
I20260812 05:25:37.495963 300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "a832ab86cff0417ba28fdaeea31d137e"
format_stamp: "Formatted at 2026-08-12 05:25:37 on dist-test-slave-zpfg"
server_key: "4e1433ffb647196a3b7ff4fcbfee8b0c"
server_key_iv: "e4bd0aa03e9e020067896501cc6db2a7"
server_key_version: "encryptionkey@0"
I20260812 05:25:37.497541 300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:37.498517 317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:37.498749 300 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:37.498852 300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "a832ab86cff0417ba28fdaeea31d137e"
format_stamp: "Formatted at 2026-08-12 05:25:37 on dist-test-slave-zpfg"
server_key: "4e1433ffb647196a3b7ff4fcbfee8b0c"
server_key_iv: "e4bd0aa03e9e020067896501cc6db2a7"
server_key_version: "encryptionkey@0"
I20260812 05:25:37.498930 300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:37.514303 300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:37.516642 300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:37.516860 300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:37.522084 300 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:38153
I20260812 05:25:37.522167 383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:38153 every 8 connection(s)
I20260812 05:25:37.522578 300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:37.523370 384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:37.525806 384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e: Bootstrap starting.
I20260812 05:25:37.526507 384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:37.526796 384 log.cc:824] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:37.527537 384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e: No bootstrap required, opened a new log
I20260812 05:25:37.527887 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 300
I20260812 05:25:37.527994 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:37.528218 24841 external_mini_cluster.cc:1428] Setting key 643e19d59c6d33401155ded695c4a126
I20260812 05:25:37.530545 384 raft_consensus.cc:348] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a832ab86cff0417ba28fdaeea31d137e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38153 } }
I20260812 05:25:37.530741 384 raft_consensus.cc:374] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:37.530783 384 raft_consensus.cc:729] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a832ab86cff0417ba28fdaeea31d137e, State: Initialized, Role: FOLLOWER
I20260812 05:25:37.530928 384 consensus_queue.cc:260] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a832ab86cff0417ba28fdaeea31d137e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38153 } }
I20260812 05:25:37.531020 384 raft_consensus.cc:388] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:37.531059 384 raft_consensus.cc:482] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:37.531121 384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:37.531888 384 raft_consensus.cc:504] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a832ab86cff0417ba28fdaeea31d137e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38153 } }
I20260812 05:25:37.531999 384 leader_election.cc:302] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a832ab86cff0417ba28fdaeea31d137e; no voters:
I20260812 05:25:37.532167 384 leader_election.cc:288] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:37.532223 391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:37.532433 384 sys_catalog.cc:564] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:37.532447 391 raft_consensus.cc:686] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [term 1 LEADER]: Becoming Leader. State: Replica: a832ab86cff0417ba28fdaeea31d137e, State: Running, Role: LEADER
I20260812 05:25:37.532558 391 consensus_queue.cc:237] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a832ab86cff0417ba28fdaeea31d137e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38153 } }
I20260812 05:25:37.532905 391 sys_catalog.cc:455] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a832ab86cff0417ba28fdaeea31d137e. Latest consensus state: current_term: 1 leader_uuid: "a832ab86cff0417ba28fdaeea31d137e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a832ab86cff0417ba28fdaeea31d137e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38153 } } }
I20260812 05:25:37.532967 391 sys_catalog.cc:458] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:37.533113 393 sys_catalog.cc:455] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a832ab86cff0417ba28fdaeea31d137e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a832ab86cff0417ba28fdaeea31d137e" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38153 } } }
I20260812 05:25:37.533171 393 sys_catalog.cc:458] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:37.533535 400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:37.534356 400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:37.536324 400 catalog_manager.cc:1347] Generated new cluster ID: 11b3c4b0f1a443e49b87a4d6745e0109
I20260812 05:25:37.536382 400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:25:37 dist-test-slave-zpfg krb5kdc[32755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:37.540848 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:37.528795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.24.66.126:38153)
0812 05:25:37.528981 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:37.528989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:37.529015 (+ 26us) server_negotiation.cc:407] Connection header received
0812 05:25:37.530044 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:37.530052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:37.530101 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:37.530226 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:37.531143 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.532528 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.533648 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.534389 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.534501 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:37.537406 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:37.537410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:37.537411 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:37.537424 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:37.539238 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:37.539622 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:37.539626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:37.539626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:37.539736 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:37.539957 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:37.539958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:37.539958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:37.540212 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:37.540400 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:37.540483 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 05:25:37.540632 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260812 05:25:37.547567 400 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:37.548166 400 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:37.551779 400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e: Generated new TSK 0
I20260812 05:25:37.551996 400 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab.
Entry for principal oryx/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab.
I20260812 05:25:37.586997 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:38153
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab
--principal=oryx/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:34341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:37.695420 414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:37.695708 414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:37.695762 414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:37.697211 414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:37.697306 414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:37.697350 414 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:25:37.697394 414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:37.699323 414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab
--principal=oryx/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.66.126:38153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:37.699661 414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:37.699952 414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:37.706036 422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:37.705978 420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:37.706058 414 server_base.cc:1034] running on GCE node
W20260812 05:25:37.705978 419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:37.706676 414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:37.707008 414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:37.708263 414 hybrid_clock.cc:648] HybridClock initialized: now 1786512337708248 us; error 29 us; skew 500 ppm
Aug 12 05:25:37 dist-test-slave-zpfg krb5kdc[32755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512337, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:37.711464 414 init.cc:377] Logged in from keytab as oryx/127.24.66.65@KRBTEST.COM (short username oryx)
I20260812 05:25:37.711963 414 webserver.cc:460] Webserver started at http://127.24.66.65:43879/ using document root <none> and password file <none>
I20260812 05:25:37.712252 414 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:37.712338 414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:37.712514 414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:37.713802 414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "a838cc64cdfc456797d7bd33be666592"
format_stamp: "Formatted at 2026-08-12 05:25:37 on dist-test-slave-zpfg"
server_key: "28e3d7458922890a57f9f347af619783"
server_key_iv: "5b5e7736acc6b5a5c4ec40d02e7f37bf"
server_key_version: "encryptionkey@0"
I20260812 05:25:37.714253 414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "a838cc64cdfc456797d7bd33be666592"
format_stamp: "Formatted at 2026-08-12 05:25:37 on dist-test-slave-zpfg"
server_key: "28e3d7458922890a57f9f347af619783"
server_key_iv: "5b5e7736acc6b5a5c4ec40d02e7f37bf"
server_key_version: "encryptionkey@0"
I20260812 05:25:37.715858 414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:37.716733 429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:37.716981 414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:37.717105 414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "a838cc64cdfc456797d7bd33be666592"
format_stamp: "Formatted at 2026-08-12 05:25:37 on dist-test-slave-zpfg"
server_key: "28e3d7458922890a57f9f347af619783"
server_key_iv: "5b5e7736acc6b5a5c4ec40d02e7f37bf"
server_key_version: "encryptionkey@0"
I20260812 05:25:37.717197 414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:37.727205 414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:37.729575 414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:37.729751 414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:37.730461 414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:37.730525 414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:37.730571 414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:37.730599 414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:37.737838 414 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:44061
I20260812 05:25:37.737980 551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:44061 every 8 connection(s)
I20260812 05:25:37.738251 414 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
Aug 12 05:25:37 dist-test-slave-zpfg krb5kdc[32755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512337, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:37.741609 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:37.730986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:46817 (local address 127.24.66.126:38153)
0812 05:25:37.731034 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:37.731036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:37.731381 (+ 345us) server_negotiation.cc:407] Connection header received
0812 05:25:37.732016 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:37.732018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:37.732031 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:37.732087 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:37.733091 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.734052 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.735848 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.736081 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.736190 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:37.739073 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:37.739074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:37.739074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:37.739082 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:37.740337 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:37.740750 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:37.740751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:37.740751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:37.740842 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:37.741074 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:37.741075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:37.741075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:37.741193 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:37.741248 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:37.741451 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 05:25:37.741543 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:25:37.741860 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:37.730965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38153 (local address 127.24.66.65:46817)
0812 05:25:37.731227 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:37.731280 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:37.731893 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:37.732172 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:37.732177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:37.732621 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:37.732971 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:37.732978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.734261 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.734262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:37.735754 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:37.735758 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.736204 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.736205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:37.736360 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:37.737076 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:37.737087 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:37.738972 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:37.740419 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:37.740426 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:37.740428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:37.740656 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:37.740911 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:37.740911 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:37.740912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:37.741013 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:37.741235 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:37.741238 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:37.741358 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 05:25:37.741436 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 05:25:37.741567 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":87,"threads_started":1}
I20260812 05:25:37.742501 552 heartbeater.cc:347] Connected to a master server at 127.24.66.126:38153
I20260812 05:25:37.742594 552 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:37.742794 552 heartbeater.cc:511] Master 127.24.66.126:38153 requested a full tablet report, sending...
I20260812 05:25:37.743471 333 ts_manager.cc:194] Registered new tserver with Master: a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061)
I20260812 05:25:37.744319 333 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:46817
I20260812 05:25:37.744360 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 414
I20260812 05:25:37.744470 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:37.744699 24841 external_mini_cluster.cc:1428] Setting key 02c9fd6fa308a3207dd3d96d854bbda9
I20260812 05:25:37.750458 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:37.743608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38153 (local address 127.24.66.65:46849)
0812 05:25:37.743825 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:37.743836 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:37.743898 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:37.744022 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:37.744022 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:37.744231 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:37.744388 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:37.744390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.745260 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.745260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:37.746060 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:37.746063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.746439 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.746440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:37.746531 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:37.746986 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:37.746992 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:37.747456 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:37.749358 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:37.749359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:37.749359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:37.749538 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:37.749748 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:37.749748 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:37.749749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:37.749800 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:37.750079 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:37.750080 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:37.750124 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 05:25:37.750175 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 05:25:37.750270 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":155,"threads_started":1}
I20260812 05:25:37.750816 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:37.743656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:46849 (local address 127.24.66.126:38153)
0812 05:25:37.743705 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:37.743707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:37.743908 (+ 201us) server_negotiation.cc:407] Connection header received
0812 05:25:37.743931 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:37.743932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:37.743951 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:37.744013 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:37.744502 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.745166 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.746140 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.746330 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.746443 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:37.747559 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:37.747560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:37.747561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:37.747569 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:37.749263 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:37.749614 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:37.749615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:37.749616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:37.749682 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:37.749863 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:37.749864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:37.749864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:37.750016 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:37.750087 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:37.750237 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 05:25:37.750333 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
WARNING: no policy specified for oryx/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab.
Entry for principal oryx/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab.
I20260812 05:25:37.778123 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:38153
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab
--principal=oryx/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:34341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:37.867055 561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:37.867295 561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:37.867323 561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:37.868642 561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:37.868696 561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:37.868719 561 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:25:37.868738 561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:37.870447 561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab
--principal=oryx/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.66.126:38153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:37.870729 561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:37.871042 561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:37.876751 567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:37.876775 569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:37.876752 566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:37.877139 561 server_base.cc:1034] running on GCE node
I20260812 05:25:37.877409 561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:37.877692 561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:37.878837 561 hybrid_clock.cc:648] HybridClock initialized: now 1786512337878749 us; error 116 us; skew 500 ppm
Aug 12 05:25:37 dist-test-slave-zpfg krb5kdc[32755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512337, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:37.881700 561 init.cc:377] Logged in from keytab as oryx/127.24.66.66@KRBTEST.COM (short username oryx)
I20260812 05:25:37.882221 561 webserver.cc:460] Webserver started at http://127.24.66.66:42921/ using document root <none> and password file <none>
I20260812 05:25:37.882531 561 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:37.882582 561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:37.882746 561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:37.883922 561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "74645a0f996c46e4bdac38c4b43fefac"
format_stamp: "Formatted at 2026-08-12 05:25:37 on dist-test-slave-zpfg"
server_key: "6dcee90f7332711ca848aaf31c220184"
server_key_iv: "478b9de96f1b96b4c1825fe4e7708f32"
server_key_version: "encryptionkey@0"
I20260812 05:25:37.884310 561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "74645a0f996c46e4bdac38c4b43fefac"
format_stamp: "Formatted at 2026-08-12 05:25:37 on dist-test-slave-zpfg"
server_key: "6dcee90f7332711ca848aaf31c220184"
server_key_iv: "478b9de96f1b96b4c1825fe4e7708f32"
server_key_version: "encryptionkey@0"
I20260812 05:25:37.885833 561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:25:37.887014 576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:37.887307 561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:25:37.887390 561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "74645a0f996c46e4bdac38c4b43fefac"
format_stamp: "Formatted at 2026-08-12 05:25:37 on dist-test-slave-zpfg"
server_key: "6dcee90f7332711ca848aaf31c220184"
server_key_iv: "478b9de96f1b96b4c1825fe4e7708f32"
server_key_version: "encryptionkey@0"
I20260812 05:25:37.887490 561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:37.913714 561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:37.916718 561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:37.916915 561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:37.917696 561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:37.917766 561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:37.917804 561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:37.917832 561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:25:37 dist-test-slave-zpfg krb5kdc[32755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512337, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:37.926237 561 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:38117
I20260812 05:25:37.926364 698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:38117 every 8 connection(s)
I20260812 05:25:37.926704 561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:37.929147 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:37.918220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:35593 (local address 127.24.66.126:38153)
0812 05:25:37.918302 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:37.918304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:37.918697 (+ 393us) server_negotiation.cc:407] Connection header received
0812 05:25:37.919513 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:37.919515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:37.919527 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:37.919577 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:37.920629 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.921311 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.922416 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.922660 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.922742 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:37.925781 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:37.925783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:37.925784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:37.925793 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:37.927580 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:37.928067 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:37.928068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:37.928069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:37.928143 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:37.928474 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:37.928476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:37.928476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:37.928608 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:37.928673 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:37.929006 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 05:25:37.929082 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 05:25:37.929378 585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:37.918195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38153 (local address 127.24.66.66:35593)
0812 05:25:37.918518 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:37.918581 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:37.919400 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:37.919674 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:37.919678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:37.920130 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:37.920484 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:37.920491 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.921413 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.921415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:37.922281 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:37.922284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.922754 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.922756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:37.922916 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:37.923605 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:37.923618 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:37.925653 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:37.927682 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:37.927689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:37.927690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:37.927984 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:37.928240 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:37.928241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:37.928242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:37.928372 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:37.928678 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:37.928683 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:37.928858 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 05:25:37.928976 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 05:25:37.929121 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":101,"threads_started":1}
I20260812 05:25:37.930059 699 heartbeater.cc:347] Connected to a master server at 127.24.66.126:38153
I20260812 05:25:37.930152 699 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:37.930426 699 heartbeater.cc:511] Master 127.24.66.126:38153 requested a full tablet report, sending...
I20260812 05:25:37.930867 333 ts_manager.cc:194] Registered new tserver with Master: 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117)
I20260812 05:25:37.931216 333 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:35593
I20260812 05:25:37.935501 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 561
I20260812 05:25:37.935642 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:37.935864 24841 external_mini_cluster.cc:1428] Setting key 47e4c32559185b36826280d936082bae
I20260812 05:25:37.938107 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:37.931356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:44605 (local address 127.24.66.126:38153)
0812 05:25:37.931420 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:37.931422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:37.931508 (+ 86us) server_negotiation.cc:407] Connection header received
0812 05:25:37.931578 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:37.931579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:37.931589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:37.931656 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:37.932060 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.932714 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.933428 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.933687 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.933763 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:37.934873 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:37.934875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:37.934875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:37.934883 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:37.936679 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:37.937146 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:37.937148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:37.937148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:37.937213 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:37.937510 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:37.937510 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:37.937511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:37.937681 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:37.937740 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:37.937956 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 05:25:37.938039 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:25:37.938197 706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:37.931278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38153 (local address 127.24.66.66:44605)
0812 05:25:37.931430 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:37.931442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:37.931526 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:37.931680 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:37.931681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:37.931893 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:37.931952 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:37.931954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.932795 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.932797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:37.933348 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:37.933350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:37.933801 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:37.933802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:37.933873 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:37.934447 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:37.934452 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:37.934764 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:37.936772 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:37.936773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:37.936774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:37.937061 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:37.937362 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:37.937363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:37.937363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:37.937444 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:37.937751 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:37.937752 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:37.937810 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 05:25:37.937894 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 05:25:37.938038 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for oryx/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab.
Entry for principal oryx/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab.
I20260812 05:25:37.966554 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:38153
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab
--principal=oryx/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:34341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:38.055953 709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:38.056205 709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:38.056234 709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:38.057480 709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:38.057529 709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:38.057551 709 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 05:25:38.057569 709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:38.059250 709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab
--principal=oryx/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.66.126:38153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:38.059523 709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:38.059773 709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:38.065940 714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:38.065966 715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:38.065940 717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:38.066318 709 server_base.cc:1034] running on GCE node
I20260812 05:25:38.066587 709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:38.066895 709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:38.068073 709 hybrid_clock.cc:648] HybridClock initialized: now 1786512338068038 us; error 64 us; skew 500 ppm
Aug 12 05:25:38 dist-test-slave-zpfg krb5kdc[32755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512338, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:38.071470 709 init.cc:377] Logged in from keytab as oryx/127.24.66.67@KRBTEST.COM (short username oryx)
I20260812 05:25:38.071979 709 webserver.cc:460] Webserver started at http://127.24.66.67:32783/ using document root <none> and password file <none>
I20260812 05:25:38.072181 709 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:38.072223 709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:38.072320 709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:38.073561 709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "2d1992b8b0374e5a874de2c92808214c"
format_stamp: "Formatted at 2026-08-12 05:25:38 on dist-test-slave-zpfg"
server_key: "69d34f67781587009d3e8a63b370cfe0"
server_key_iv: "a769104975aff42e92fc52d5cf57b1f2"
server_key_version: "encryptionkey@0"
I20260812 05:25:38.073891 709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "2d1992b8b0374e5a874de2c92808214c"
format_stamp: "Formatted at 2026-08-12 05:25:38 on dist-test-slave-zpfg"
server_key: "69d34f67781587009d3e8a63b370cfe0"
server_key_iv: "a769104975aff42e92fc52d5cf57b1f2"
server_key_version: "encryptionkey@0"
I20260812 05:25:38.075490 709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:38.076488 724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:38.076756 709 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:38.076880 709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "2d1992b8b0374e5a874de2c92808214c"
format_stamp: "Formatted at 2026-08-12 05:25:38 on dist-test-slave-zpfg"
server_key: "69d34f67781587009d3e8a63b370cfe0"
server_key_iv: "a769104975aff42e92fc52d5cf57b1f2"
server_key_version: "encryptionkey@0"
I20260812 05:25:38.076963 709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:38.094550 709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:38.096912 709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:38.097111 709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:38.097703 709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:38.097774 709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:38.097812 709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:38.097841 709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 05:25:38 dist-test-slave-zpfg krb5kdc[32755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512338, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:38.105898 709 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:35143
I20260812 05:25:38.106269 709 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:38.106304 845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:35143 every 8 connection(s)
I20260812 05:25:38.108687 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.098265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:49507 (local address 127.24.66.126:38153)
0812 05:25:38.098368 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.098370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.098597 (+ 227us) server_negotiation.cc:407] Connection header received
0812 05:25:38.099336 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.099337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.099367 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.099448 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:38.100440 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.100973 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.102741 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.103036 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.103122 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.105406 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:38.105407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:38.105408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:38.105416 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:38.107056 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:38.107572 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:38.107574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:38.107574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:38.107652 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:38.107914 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:38.107915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:38.107916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:38.108098 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:38.108193 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.108509 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.108594 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 05:25:38.108886 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.098230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38153 (local address 127.24.66.67:49507)
0812 05:25:38.098433 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.098487 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.099210 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.099555 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.099560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.099999 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:38.100319 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.100326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.101058 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.101059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.102645 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.102649 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.103144 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.103145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.103290 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.103859 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:38.103867 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:38.105292 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:38.107146 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:38.107152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:38.107155 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:38.107482 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:38.107726 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:38.107727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:38.107727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:38.107840 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:38.108185 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:38.108193 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:38.108368 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.108484 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.108648 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":115,"threads_started":1}
I20260812 05:25:38.109592 846 heartbeater.cc:347] Connected to a master server at 127.24.66.126:38153
I20260812 05:25:38.109691 846 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:38.109925 846 heartbeater.cc:511] Master 127.24.66.126:38153 requested a full tablet report, sending...
I20260812 05:25:38.110378 334 ts_manager.cc:194] Registered new tserver with Master: 2d1992b8b0374e5a874de2c92808214c (127.24.66.67:35143)
I20260812 05:25:38.110790 334 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:49507
I20260812 05:25:38.113094 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 709
I20260812 05:25:38.113215 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:38.113467 24841 external_mini_cluster.cc:1428] Setting key 43f9654d523fad2ab714a049995ae5ca
I20260812 05:25:38.114450 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:25:38.117617 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.111478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38153 (local address 127.24.66.67:38003)
0812 05:25:38.111665 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.111675 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.111752 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.111883 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.111883 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.112069 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:38.112126 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.112128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.112908 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.112909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.113533 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.113535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.113955 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.113956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.114032 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.114558 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:38.114564 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:38.114879 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:38.116609 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:38.116610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:38.116611 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:38.116806 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:38.117034 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:38.117034 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:38.117035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:38.117089 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:38.117370 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:38.117371 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:38.117412 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.117464 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.117546 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
I20260812 05:25:38.117683 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.111558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:38003 (local address 127.24.66.126:38153)
0812 05:25:38.111591 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.111593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.111736 (+ 143us) server_negotiation.cc:407] Connection header received
0812 05:25:38.111810 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.111811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.111821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.111876 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:38.112296 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.112802 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.113627 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.113868 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.113953 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.114972 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:38.114974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:38.114974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:38.114981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:38.116500 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:38.116889 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:38.116890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:38.116891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:38.116963 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:38.117153 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:38.117153 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:38.117154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:38.117303 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:38.117378 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.117494 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.117585 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
Aug 12 05:25:38 dist-test-slave-zpfg krb5kdc[32755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512338, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 05:25:38 dist-test-slave-zpfg krb5kdc[32755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:38 dist-test-slave-zpfg krb5kdc[32755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 05:25:38.139560 390 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:38.133234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41498 (local address 127.24.66.126:38153)
0812 05:25:38.133334 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.133336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.133346 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:38.133450 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.133451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.133462 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.133567 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:38.134247 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.134821 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.135680 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.135900 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.135977 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.139494 (+ 3517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41498: BlockingRecv error: recv got EOF from 127.0.0.1:41498 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 05:25:38 dist-test-slave-zpfg krb5kdc[32755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512338, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:38.148139 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.140702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41510 (local address 127.24.66.126:38153)
0812 05:25:38.140782 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.140784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.140813 (+ 29us) server_negotiation.cc:407] Connection header received
0812 05:25:38.140876 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.140877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.140886 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.140938 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:38.141414 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.142010 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.142844 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.143069 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.143127 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.145385 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:38.145386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:38.145387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:38.145394 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:38.146951 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:38.147322 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:38.147323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:38.147324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:38.147392 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:38.147629 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:38.147631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:38.147631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:38.147778 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:38.147878 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.147992 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.148065 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:25:38.150080 334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:38.151868 334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:38.159075 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.155044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60956 (local address 127.24.66.67:35143)
0812 05:25:38.155240 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.155245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.155259 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:25:38.155346 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.155349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.155367 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.155445 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:38.155711 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.156467 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.157819 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.158591 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.158684 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.158779 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.158931 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.158970 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
I20260812 05:25:38.159068 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.154774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:35143 (local address 127.0.0.1:60956)
0812 05:25:38.155202 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.155214 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.155269 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.155537 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.155541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.155543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:38.155618 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.155620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.156562 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.156563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.157728 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.157730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.158683 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.158684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.158783 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.158807 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.158911 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.158972 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":105,"threads_started":1}
I20260812 05:25:38.159440 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.154828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:38117 (local address 127.0.0.1:36798)
0812 05:25:38.155265 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.155276 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.155325 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.155671 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.155672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.155674 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:38.155711 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.155712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.157067 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.157068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.158067 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.158069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.159170 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.159171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.159251 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.159257 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.159308 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.159354 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":30,"thread_start_us":73,"threads_started":1}
I20260812 05:25:38.159557 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.154770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:44061 (local address 127.0.0.1:41194)
0812 05:25:38.154893 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.154935 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.155007 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.155672 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.155674 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.155676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:38.155744 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.155746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.157096 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.157097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.158239 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.158241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.159341 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.159342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.159390 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.159394 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.159422 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.159453 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":27,"thread_start_us":77,"threads_started":1}
I20260812 05:25:38.159863 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.155080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36798 (local address 127.24.66.66:38117)
0812 05:25:38.155457 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.155462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.155470 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:38.155508 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.155510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.155525 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.155593 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:38.155777 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.156967 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.158185 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.158976 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.159050 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.159129 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.159713 (+ 584us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.159763 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":92,"threads_started":1}
I20260812 05:25:38.159909 776 tablet_service.cc:1467] Processing CreateTablet for tablet 7e42691b275c44e88cea96dde0e6964c (DEFAULT_TABLE table=test-table [id=318c55f3ed0d4e958e87bd9710ee4547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:38.160073 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.155002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.24.66.65:44061)
0812 05:25:38.155427 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.155431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.155442 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:38.155484 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.155486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.155502 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.155830 (+ 328us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:38.155897 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.156991 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.158325 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.159085 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.159497 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.159903 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.159931 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.159981 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":84,"threads_started":1}
I20260812 05:25:38.160344 776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e42691b275c44e88cea96dde0e6964c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.160496 629 tablet_service.cc:1467] Processing CreateTablet for tablet 7e42691b275c44e88cea96dde0e6964c (DEFAULT_TABLE table=test-table [id=318c55f3ed0d4e958e87bd9710ee4547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:38.160611 482 tablet_service.cc:1467] Processing CreateTablet for tablet 7e42691b275c44e88cea96dde0e6964c (DEFAULT_TABLE table=test-table [id=318c55f3ed0d4e958e87bd9710ee4547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:38.160885 629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e42691b275c44e88cea96dde0e6964c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.160907 482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e42691b275c44e88cea96dde0e6964c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.163549 876 tablet_bootstrap.cc:492] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c: Bootstrap starting.
I20260812 05:25:38.164324 876 tablet_bootstrap.cc:654] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.164696 876 log.cc:824] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:38.164762 877 tablet_bootstrap.cc:492] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: Bootstrap starting.
I20260812 05:25:38.165388 877 tablet_bootstrap.cc:654] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.165627 877 log.cc:824] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:38.166322 877 tablet_bootstrap.cc:492] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: No bootstrap required, opened a new log
I20260812 05:25:38.166445 877 ts_tablet_manager.cc:1397] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:25:38.167050 876 tablet_bootstrap.cc:492] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c: No bootstrap required, opened a new log
I20260812 05:25:38.167136 876 ts_tablet_manager.cc:1397] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 05:25:38.167105 880 tablet_bootstrap.cc:492] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: Bootstrap starting.
I20260812 05:25:38.167773 880 tablet_bootstrap.cc:654] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.168105 880 log.cc:824] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:38.168960 880 tablet_bootstrap.cc:492] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: No bootstrap required, opened a new log
I20260812 05:25:38.169052 880 ts_tablet_manager.cc:1397] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:25:38.168936 876 raft_consensus.cc:348] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } }
I20260812 05:25:38.168989 877 raft_consensus.cc:348] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } }
I20260812 05:25:38.169128 876 raft_consensus.cc:374] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.169169 876 raft_consensus.cc:729] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1992b8b0374e5a874de2c92808214c, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.169173 877 raft_consensus.cc:374] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.169210 877 raft_consensus.cc:729] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a838cc64cdfc456797d7bd33be666592, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.169294 876 consensus_queue.cc:260] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } }
I20260812 05:25:38.169328 877 consensus_queue.cc:260] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } }
I20260812 05:25:38.169620 877 ts_tablet_manager.cc:1428] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 05:25:38.169620 876 ts_tablet_manager.cc:1428] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:25:38.169678 552 heartbeater.cc:503] Master 127.24.66.126:38153 was elected leader, sending a full tablet report...
I20260812 05:25:38.169765 846 heartbeater.cc:503] Master 127.24.66.126:38153 was elected leader, sending a full tablet report...
I20260812 05:25:38.171260 880 raft_consensus.cc:348] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } }
I20260812 05:25:38.171393 880 raft_consensus.cc:374] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.171415 880 raft_consensus.cc:729] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74645a0f996c46e4bdac38c4b43fefac, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.171535 880 consensus_queue.cc:260] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } }
I20260812 05:25:38.171765 880 ts_tablet_manager.cc:1428] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:25:38.171864 699 heartbeater.cc:503] Master 127.24.66.126:38153 was elected leader, sending a full tablet report...
W20260812 05:25:38.177397 701 tablet.cc:2369] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:25:38.239131 553 tablet.cc:2369] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:25:38.324852 883 raft_consensus.cc:482] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:38.325030 883 raft_consensus.cc:504] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } }
I20260812 05:25:38.325480 883 leader_election.cc:288] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061), 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117)
I20260812 05:25:38.328555 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.325767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:60231 (local address 127.24.66.65:44061)
0812 05:25:38.325818 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.325820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.325832 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:25:38.325954 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.325955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.325968 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.326028 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:38.326232 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.326815 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.327787 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.328173 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.328229 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.328235 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.328432 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.328477 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:25:38.328568 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.325886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:60533 (local address 127.24.66.66:38117)
0812 05:25:38.325936 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.325938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.326040 (+ 102us) server_negotiation.cc:407] Connection header received
0812 05:25:38.326071 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.326072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.326081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.326143 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:38.326404 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.326910 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.327852 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.328224 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.328300 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.328308 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.328458 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.328496 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:25:38.328590 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.325700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:44061 (local address 127.24.66.67:60231)
0812 05:25:38.325762 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.325776 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.325865 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.326059 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.326060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.326062 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:38.326143 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.326145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.326897 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.326898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.327693 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.327695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.328255 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.328256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.328350 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.328354 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.328413 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.328480 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 05:25:38.328578 886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.325765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:38117 (local address 127.24.66.67:60533)
0812 05:25:38.325938 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.325949 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.326018 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.326231 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.326232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.326234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:38.326303 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.326304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.326997 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.326997 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.327724 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.327726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.328312 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.328313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.328367 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.328370 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.328444 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.328481 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":106,"threads_started":1}
I20260812 05:25:38.328974 502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e42691b275c44e88cea96dde0e6964c" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a838cc64cdfc456797d7bd33be666592" is_pre_election: true
I20260812 05:25:38.329172 502 raft_consensus.cc:2454] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 0.
I20260812 05:25:38.329128 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e42691b275c44e88cea96dde0e6964c" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74645a0f996c46e4bdac38c4b43fefac" is_pre_election: true
I20260812 05:25:38.329312 649 raft_consensus.cc:2454] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 0.
I20260812 05:25:38.329456 726 leader_election.cc:302] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1992b8b0374e5a874de2c92808214c, a838cc64cdfc456797d7bd33be666592; no voters:
I20260812 05:25:38.329586 883 raft_consensus.cc:2781] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:38.329640 883 raft_consensus.cc:482] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:38.329667 883 raft_consensus.cc:3037] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:38.330778 883 raft_consensus.cc:504] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } }
I20260812 05:25:38.330935 883 leader_election.cc:288] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 election: Requested vote from peers a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061), 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117)
I20260812 05:25:38.331089 502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e42691b275c44e88cea96dde0e6964c" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a838cc64cdfc456797d7bd33be666592"
I20260812 05:25:38.331106 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e42691b275c44e88cea96dde0e6964c" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74645a0f996c46e4bdac38c4b43fefac"
I20260812 05:25:38.331184 649 raft_consensus.cc:3037] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:38.331175 502 raft_consensus.cc:3037] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:38.332075 649 raft_consensus.cc:2454] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 1.
I20260812 05:25:38.332075 502 raft_consensus.cc:2454] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 1.
I20260812 05:25:38.332273 726 leader_election.cc:302] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1992b8b0374e5a874de2c92808214c, a838cc64cdfc456797d7bd33be666592; no voters:
I20260812 05:25:38.332379 883 raft_consensus.cc:2781] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:38.332556 883 raft_consensus.cc:686] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 1 LEADER]: Becoming Leader. State: Replica: 2d1992b8b0374e5a874de2c92808214c, State: Running, Role: LEADER
I20260812 05:25:38.332674 883 consensus_queue.cc:237] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } }
I20260812 05:25:38.333760 334 catalog_manager.cc:5495] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d1992b8b0374e5a874de2c92808214c (127.24.66.67). New cstate: current_term: 1 leader_uuid: "2d1992b8b0374e5a874de2c92808214c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } health_report { overall_health: UNKNOWN } } }
I20260812 05:25:38.345162 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.342860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.24.66.67:35143)
0812 05:25:38.342920 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.342922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.342932 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:38.342978 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.342979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.342998 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.343061 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:38.343211 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.343767 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.344494 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.344679 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.344731 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.344887 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:38.344988 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:38.345050 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.345081 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.345101 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 05:25:38.347121 649 raft_consensus.cc:1261] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Refusing update from remote peer 2d1992b8b0374e5a874de2c92808214c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:38.347149 502 raft_consensus.cc:1261] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Refusing update from remote peer 2d1992b8b0374e5a874de2c92808214c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:38.347543 883 consensus_queue.cc:1035] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:38.347736 887 consensus_queue.cc:1035] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:38.351228 892 mvcc.cc:205] Tried to move back new op lower bound from 7317554537867059200 to 7317554537811537920. Current Snapshot: MvccSnapshot[applied={T|T < 7317554537867059200}]
I20260812 05:25:38.352644 333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:38.352761 333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:38.353632 333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e: Sending DeleteTablet for 3 replicas of tablet 7e42691b275c44e88cea96dde0e6964c
I20260812 05:25:38.354017 629 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e42691b275c44e88cea96dde0e6964c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:38 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:36798
I20260812 05:25:38.354013 482 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e42691b275c44e88cea96dde0e6964c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:38 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:41194
I20260812 05:25:38.354043 776 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e42691b275c44e88cea96dde0e6964c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:38 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:60956
I20260812 05:25:38.354322 904 tablet_replica.cc:330] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c: stopping tablet replica
I20260812 05:25:38.354465 904 raft_consensus.cc:2229] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:38.354598 904 raft_consensus.cc:2258] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:38.354483 333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:38.354687 333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:38.354956 902 tablet_replica.cc:330] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: stopping tablet replica
I20260812 05:25:38.355361 904 ts_tablet_manager.cc:1905] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:38.355803 902 raft_consensus.cc:2229] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:38.355949 902 raft_consensus.cc:2258] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:38.356029 482 tablet_service.cc:1467] Processing CreateTablet for tablet 4218e2c09f17449b92f7cf02fcb30210 (DEFAULT_TABLE table=test-table [id=1e8454c46b0240278bcb33cf75243402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:38.356109 629 tablet_service.cc:1467] Processing CreateTablet for tablet 4218e2c09f17449b92f7cf02fcb30210 (DEFAULT_TABLE table=test-table [id=1e8454c46b0240278bcb33cf75243402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:38.356174 482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4218e2c09f17449b92f7cf02fcb30210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.356208 776 tablet_service.cc:1467] Processing CreateTablet for tablet 4218e2c09f17449b92f7cf02fcb30210 (DEFAULT_TABLE table=test-table [id=1e8454c46b0240278bcb33cf75243402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:38.356238 629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4218e2c09f17449b92f7cf02fcb30210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.356313 776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4218e2c09f17449b92f7cf02fcb30210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.356384 903 tablet_replica.cc:330] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: stopping tablet replica
I20260812 05:25:38.356475 903 raft_consensus.cc:2229] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:38.356571 903 raft_consensus.cc:2258] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:38.357126 902 ts_tablet_manager.cc:1905] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:38.357635 876 tablet_bootstrap.cc:492] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c: Bootstrap starting.
I20260812 05:25:38.357657 877 tablet_bootstrap.cc:492] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592: Bootstrap starting.
I20260812 05:25:38.358068 877 tablet_bootstrap.cc:654] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.358068 876 tablet_bootstrap.cc:654] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.358464 880 tablet_bootstrap.cc:492] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac: Bootstrap starting.
I20260812 05:25:38.358850 876 tablet_bootstrap.cc:492] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c: No bootstrap required, opened a new log
I20260812 05:25:38.358896 876 ts_tablet_manager.cc:1397] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:38.358925 902 ts_tablet_manager.cc:1918] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:38.358999 902 log.cc:1196] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/wal/wals/7e42691b275c44e88cea96dde0e6964c
I20260812 05:25:38.358999 880 tablet_bootstrap.cc:654] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.359046 876 raft_consensus.cc:348] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.359113 876 raft_consensus.cc:374] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.359133 876 raft_consensus.cc:729] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1992b8b0374e5a874de2c92808214c, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.359177 876 consensus_queue.cc:260] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.359273 876 ts_tablet_manager.cc:1428] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:38.359447 902 ts_tablet_manager.cc:1939] T 7e42691b275c44e88cea96dde0e6964c P a838cc64cdfc456797d7bd33be666592: Deleting consensus metadata
I20260812 05:25:38.360071 319 catalog_manager.cc:4841] TS a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061): tablet 7e42691b275c44e88cea96dde0e6964c (table test-table [id=318c55f3ed0d4e958e87bd9710ee4547]) successfully deleted
I20260812 05:25:38.360205 903 ts_tablet_manager.cc:1905] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:38.361232 904 ts_tablet_manager.cc:1918] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:38.361385 904 log.cc:1196] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/7e42691b275c44e88cea96dde0e6964c
I20260812 05:25:38.361444 877 tablet_bootstrap.cc:492] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592: No bootstrap required, opened a new log
I20260812 05:25:38.361503 877 ts_tablet_manager.cc:1397] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260812 05:25:38.361663 877 raft_consensus.cc:348] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.361776 877 raft_consensus.cc:374] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.361809 877 raft_consensus.cc:729] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a838cc64cdfc456797d7bd33be666592, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.361824 903 ts_tablet_manager.cc:1918] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:38.361855 877 consensus_queue.cc:260] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.361903 903 log.cc:1196] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/7e42691b275c44e88cea96dde0e6964c
I20260812 05:25:38.362002 904 ts_tablet_manager.cc:1939] T 7e42691b275c44e88cea96dde0e6964c P 2d1992b8b0374e5a874de2c92808214c: Deleting consensus metadata
I20260812 05:25:38.362205 903 ts_tablet_manager.cc:1939] T 7e42691b275c44e88cea96dde0e6964c P 74645a0f996c46e4bdac38c4b43fefac: Deleting consensus metadata
I20260812 05:25:38.362248 877 ts_tablet_manager.cc:1428] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:38.362656 321 catalog_manager.cc:4841] TS 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117): tablet 7e42691b275c44e88cea96dde0e6964c (table test-table [id=318c55f3ed0d4e958e87bd9710ee4547]) successfully deleted
I20260812 05:25:38.362970 320 catalog_manager.cc:4841] TS 2d1992b8b0374e5a874de2c92808214c (127.24.66.67:35143): tablet 7e42691b275c44e88cea96dde0e6964c (table test-table [id=318c55f3ed0d4e958e87bd9710ee4547]) successfully deleted
I20260812 05:25:38.363128 880 tablet_bootstrap.cc:492] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac: No bootstrap required, opened a new log
I20260812 05:25:38.363209 880 ts_tablet_manager.cc:1397] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 05:25:38.363390 880 raft_consensus.cc:348] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.363474 880 raft_consensus.cc:374] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.363503 880 raft_consensus.cc:729] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74645a0f996c46e4bdac38c4b43fefac, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.363559 880 consensus_queue.cc:260] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.363662 880 ts_tablet_manager.cc:1428] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:38.514885 896 raft_consensus.cc:482] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:38.515003 896 raft_consensus.cc:504] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.515250 896 leader_election.cc:288] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061), 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117)
I20260812 05:25:38.515554 502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4218e2c09f17449b92f7cf02fcb30210" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a838cc64cdfc456797d7bd33be666592" is_pre_election: true
I20260812 05:25:38.515554 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4218e2c09f17449b92f7cf02fcb30210" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74645a0f996c46e4bdac38c4b43fefac" is_pre_election: true
I20260812 05:25:38.515671 502 raft_consensus.cc:2454] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 0.
I20260812 05:25:38.515671 649 raft_consensus.cc:2454] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 0.
I20260812 05:25:38.515884 728 leader_election.cc:302] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1992b8b0374e5a874de2c92808214c, 74645a0f996c46e4bdac38c4b43fefac; no voters:
I20260812 05:25:38.516026 896 raft_consensus.cc:2781] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:38.516072 896 raft_consensus.cc:482] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:38.516086 896 raft_consensus.cc:3037] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:38.516835 896 raft_consensus.cc:504] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.516997 896 leader_election.cc:288] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 election: Requested vote from peers a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061), 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117)
I20260812 05:25:38.517196 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4218e2c09f17449b92f7cf02fcb30210" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74645a0f996c46e4bdac38c4b43fefac"
I20260812 05:25:38.517206 502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4218e2c09f17449b92f7cf02fcb30210" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a838cc64cdfc456797d7bd33be666592"
I20260812 05:25:38.517314 502 raft_consensus.cc:3037] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:38.517326 649 raft_consensus.cc:3037] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:38.518116 502 raft_consensus.cc:2454] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 1.
I20260812 05:25:38.518112 649 raft_consensus.cc:2454] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 1.
I20260812 05:25:38.518389 728 leader_election.cc:302] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1992b8b0374e5a874de2c92808214c, 74645a0f996c46e4bdac38c4b43fefac; no voters:
I20260812 05:25:38.518502 896 raft_consensus.cc:2781] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:38.518554 896 raft_consensus.cc:686] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 1 LEADER]: Becoming Leader. State: Replica: 2d1992b8b0374e5a874de2c92808214c, State: Running, Role: LEADER
I20260812 05:25:38.518610 896 consensus_queue.cc:237] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.519285 333 catalog_manager.cc:5495] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d1992b8b0374e5a874de2c92808214c (127.24.66.67). New cstate: current_term: 1 leader_uuid: "2d1992b8b0374e5a874de2c92808214c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } health_report { overall_health: HEALTHY } } }
Aug 12 05:25:38 dist-test-slave-zpfg krb5kdc[32755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512337, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.126@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:38.570119 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.563201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38153 (local address 127.0.0.1:41522)
0812 05:25:38.563437 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.563448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.563518 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.563659 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.563659 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.563992 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:38.564043 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.564045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.564740 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.564740 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.565287 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.565289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.565605 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.565606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.565646 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.566172 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:38.566176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:38.567682 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:38.569114 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:38.569122 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:38.569124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:38.569345 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:38.569521 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:38.569522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:38.569522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:38.569581 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:38.569801 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:38.569807 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:38.569856 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.569911 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.570019 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":2,"thread_start_us":118,"threads_started":1}
I20260812 05:25:38.570165 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.563367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.24.66.126:38153)
0812 05:25:38.563459 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.563461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.563515 (+ 54us) server_negotiation.cc:407] Connection header received
0812 05:25:38.563570 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.563570 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.563580 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.563634 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:38.564126 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.564636 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.565360 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.565543 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.565592 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.567798 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:38.567799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:38.567799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:38.567805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:38.569037 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:38.569409 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:38.569410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:38.569410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:38.569460 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:38.569635 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:38.569635 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:38.569635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:38.569736 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:38.569806 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.569938 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.570066 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":67}
I20260812 05:25:38.572507 333 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:41522:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 05:25:38.572705 333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:38.574370 629 tablet_service.cc:1467] Processing CreateTablet for tablet 64f6f93812444112a6e14c218934c4cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c7ce16769c074fb493ea4a0f40dc8186]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:25:38.574422 482 tablet_service.cc:1467] Processing CreateTablet for tablet 64f6f93812444112a6e14c218934c4cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c7ce16769c074fb493ea4a0f40dc8186]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:25:38.574494 776 tablet_service.cc:1467] Processing CreateTablet for tablet 64f6f93812444112a6e14c218934c4cd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c7ce16769c074fb493ea4a0f40dc8186]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 05:25:38.574555 629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64f6f93812444112a6e14c218934c4cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.574594 482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64f6f93812444112a6e14c218934c4cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.574620 776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64f6f93812444112a6e14c218934c4cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.575986 880 tablet_bootstrap.cc:492] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac: Bootstrap starting.
I20260812 05:25:38.576159 877 tablet_bootstrap.cc:492] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: Bootstrap starting.
I20260812 05:25:38.576159 876 tablet_bootstrap.cc:492] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c: Bootstrap starting.
I20260812 05:25:38.576449 880 tablet_bootstrap.cc:654] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.576517 876 tablet_bootstrap.cc:654] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.576582 877 tablet_bootstrap.cc:654] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.577204 876 tablet_bootstrap.cc:492] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c: No bootstrap required, opened a new log
I20260812 05:25:38.577284 876 ts_tablet_manager.cc:1397] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:38.577458 876 raft_consensus.cc:348] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.577520 877 tablet_bootstrap.cc:492] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: No bootstrap required, opened a new log
I20260812 05:25:38.577570 876 raft_consensus.cc:374] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.577584 877 ts_tablet_manager.cc:1397] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:38.577600 876 raft_consensus.cc:729] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1992b8b0374e5a874de2c92808214c, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.577647 876 consensus_queue.cc:260] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.577708 880 tablet_bootstrap.cc:492] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac: No bootstrap required, opened a new log
I20260812 05:25:38.577759 876 ts_tablet_manager.cc:1428] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:25:38.577767 880 ts_tablet_manager.cc:1397] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:25:38.577736 877 raft_consensus.cc:348] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.577821 877 raft_consensus.cc:374] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.577848 877 raft_consensus.cc:729] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a838cc64cdfc456797d7bd33be666592, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.577898 877 consensus_queue.cc:260] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.577919 880 raft_consensus.cc:348] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.578018 880 raft_consensus.cc:374] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.578022 877 ts_tablet_manager.cc:1428] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 05:25:38.578042 880 raft_consensus.cc:729] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74645a0f996c46e4bdac38c4b43fefac, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.578090 880 consensus_queue.cc:260] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.578199 880 ts_tablet_manager.cc:1428] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:38.578188 882 tablet_replica.cc:439] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } }
I20260812 05:25:38.578293 882 tablet_replica.cc:442] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:38.578306 884 tablet_replica.cc:439] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } }
I20260812 05:25:38.578425 884 tablet_replica.cc:442] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:38.578537 896 tablet_replica.cc:439] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } }
I20260812 05:25:38.578637 896 tablet_replica.cc:442] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 05:25:38.607841 847 tablet.cc:2369] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:25:38.823144 882 raft_consensus.cc:482] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:38.823272 882 raft_consensus.cc:504] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.823660 882 leader_election.cc:288] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117), 2d1992b8b0374e5a874de2c92808214c (127.24.66.67:35143)
I20260812 05:25:38.826944 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.824011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:41529 (local address 127.24.66.67:35143)
0812 05:25:38.824082 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.824085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.824269 (+ 184us) server_negotiation.cc:407] Connection header received
0812 05:25:38.824386 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.824387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.824407 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.824469 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:38.824622 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.825096 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.826093 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.826553 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.826643 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.826653 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.826815 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.826861 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:25:38.826980 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.824095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:46005 (local address 127.24.66.66:38117)
0812 05:25:38.824175 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.824178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.824217 (+ 39us) server_negotiation.cc:407] Connection header received
0812 05:25:38.824318 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.824319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.824329 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.824402 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:38.824549 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.825034 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.826086 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.826568 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.826661 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.826671 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.826864 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.826906 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 05:25:38.827018 921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.823908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:38117 (local address 127.24.66.65:46005)
0812 05:25:38.824144 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.824160 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.824235 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.824410 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.824411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.824413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:38.824480 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.824482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.825126 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.825127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.825985 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.825987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.826662 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.826663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.826774 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.826777 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.826853 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.826896 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":40,"thread_start_us":117,"threads_started":1}
I20260812 05:25:38.827199 920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.823908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:35143 (local address 127.24.66.65:41529)
0812 05:25:38.824177 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.824187 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.824282 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.824489 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.824490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.824492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:38.824549 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.824551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.825207 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.825208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.825989 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.825991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.826641 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.826642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.826738 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.826741 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.826820 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.826861 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":3,"thread_start_us":117,"threads_started":1}
I20260812 05:25:38.827279 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64f6f93812444112a6e14c218934c4cd" candidate_uuid: "a838cc64cdfc456797d7bd33be666592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74645a0f996c46e4bdac38c4b43fefac" is_pre_election: true
I20260812 05:25:38.827416 649 raft_consensus.cc:2454] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a838cc64cdfc456797d7bd33be666592 in term 0.
I20260812 05:25:38.827555 787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64f6f93812444112a6e14c218934c4cd" candidate_uuid: "a838cc64cdfc456797d7bd33be666592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1992b8b0374e5a874de2c92808214c" is_pre_election: true
I20260812 05:25:38.827617 434 leader_election.cc:302] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74645a0f996c46e4bdac38c4b43fefac, a838cc64cdfc456797d7bd33be666592; no voters:
I20260812 05:25:38.827721 787 raft_consensus.cc:2454] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a838cc64cdfc456797d7bd33be666592 in term 0.
I20260812 05:25:38.827832 882 raft_consensus.cc:2781] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:38.827881 882 raft_consensus.cc:482] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:38.827914 882 raft_consensus.cc:3037] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:38.828662 882 raft_consensus.cc:504] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.828837 882 leader_election.cc:288] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [CANDIDATE]: Term 1 election: Requested vote from peers 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117), 2d1992b8b0374e5a874de2c92808214c (127.24.66.67:35143)
I20260812 05:25:38.829027 649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64f6f93812444112a6e14c218934c4cd" candidate_uuid: "a838cc64cdfc456797d7bd33be666592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74645a0f996c46e4bdac38c4b43fefac"
I20260812 05:25:38.829097 787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64f6f93812444112a6e14c218934c4cd" candidate_uuid: "a838cc64cdfc456797d7bd33be666592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1992b8b0374e5a874de2c92808214c"
I20260812 05:25:38.829137 649 raft_consensus.cc:3037] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:38.829167 787 raft_consensus.cc:3037] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:38.829972 649 raft_consensus.cc:2454] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a838cc64cdfc456797d7bd33be666592 in term 1.
I20260812 05:25:38.830016 787 raft_consensus.cc:2454] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a838cc64cdfc456797d7bd33be666592 in term 1.
I20260812 05:25:38.830161 433 leader_election.cc:302] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1992b8b0374e5a874de2c92808214c, a838cc64cdfc456797d7bd33be666592; no voters:
I20260812 05:25:38.830253 882 raft_consensus.cc:2781] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:38.830422 882 raft_consensus.cc:686] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [term 1 LEADER]: Becoming Leader. State: Replica: a838cc64cdfc456797d7bd33be666592, State: Running, Role: LEADER
I20260812 05:25:38.830523 882 consensus_queue.cc:237] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.830906 922 tablet_replica.cc:439] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: TxnStatusTablet state changed. Reason: New leader a838cc64cdfc456797d7bd33be666592. Latest consensus state: current_term: 1 leader_uuid: "a838cc64cdfc456797d7bd33be666592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } }
I20260812 05:25:38.831008 922 tablet_replica.cc:442] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:25:38.831190 924 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 05:25:38.831334 924 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 05:25:38.831518 334 catalog_manager.cc:5495] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 reported cstate change: term changed from 0 to 1, leader changed from <none> to a838cc64cdfc456797d7bd33be666592 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "a838cc64cdfc456797d7bd33be666592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } health_report { overall_health: UNKNOWN } } }
I20260812 05:25:38.865063 927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.862128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41210 (local address 127.24.66.65:44061)
0812 05:25:38.862276 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.862279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.862287 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:38.862406 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.862407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.862435 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.862521 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:38.862661 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.863361 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.864157 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.864377 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.864470 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.864680 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:38.864796 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:38.864875 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.864929 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.864952 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260812 05:25:38.865056 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.862056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:44061 (local address 127.0.0.1:41210)
0812 05:25:38.862150 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.862160 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.862218 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.862527 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.862528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.862529 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:25:38.862590 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.862593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.863452 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.863454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.864064 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.864066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.864473 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.864474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.864536 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.864555 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:25:38.864869 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:25:38.864880 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.864926 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.864984 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 05:25:38.866441 649 raft_consensus.cc:1261] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Refusing update from remote peer a838cc64cdfc456797d7bd33be666592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:38.866503 787 raft_consensus.cc:1261] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Refusing update from remote peer a838cc64cdfc456797d7bd33be666592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:38.866751 882 consensus_queue.cc:1035] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:38.866964 882 consensus_queue.cc:1035] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:38.867726 896 tablet_replica.cc:439] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c: TxnStatusTablet state changed. Reason: New leader a838cc64cdfc456797d7bd33be666592. Latest consensus state: current_term: 1 leader_uuid: "a838cc64cdfc456797d7bd33be666592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } }
I20260812 05:25:38.867823 896 tablet_replica.cc:442] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:38.867980 884 tablet_replica.cc:439] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac: TxnStatusTablet state changed. Reason: New leader a838cc64cdfc456797d7bd33be666592. Latest consensus state: current_term: 1 leader_uuid: "a838cc64cdfc456797d7bd33be666592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } }
I20260812 05:25:38.868072 884 tablet_replica.cc:442] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:38.868142 922 tablet_replica.cc:439] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a838cc64cdfc456797d7bd33be666592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } }
I20260812 05:25:38.868224 922 tablet_replica.cc:442] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:25:38.868585 929 mvcc.cc:205] Tried to move back new op lower bound from 7317554539994714112 to 7317554539850469376. Current Snapshot: MvccSnapshot[applied={T|T < 7317554539994714112}]
I20260812 05:25:38.868784 884 tablet_replica.cc:439] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a838cc64cdfc456797d7bd33be666592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } }
I20260812 05:25:38.868886 884 tablet_replica.cc:442] T 64f6f93812444112a6e14c218934c4cd P 74645a0f996c46e4bdac38c4b43fefac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:38.870230 882 tablet_replica.cc:439] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a838cc64cdfc456797d7bd33be666592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } }
I20260812 05:25:38.870836 896 tablet_replica.cc:439] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a838cc64cdfc456797d7bd33be666592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } } }
I20260812 05:25:38.870924 896 tablet_replica.cc:442] T 64f6f93812444112a6e14c218934c4cd P 2d1992b8b0374e5a874de2c92808214c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 05:25:38.870329 882 tablet_replica.cc:442] T 64f6f93812444112a6e14c218934c4cd P a838cc64cdfc456797d7bd33be666592: This TxnStatusTablet replica's current role is: LEADER
I20260812 05:25:38.871246 931 mvcc.cc:205] Tried to move back new op lower bound from 7317554539994714112 to 7317554539850469376. Current Snapshot: MvccSnapshot[applied={T|T < 7317554539994714112}]
I20260812 05:25:38.875522 927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.873103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:42649 (local address 127.24.66.65:44061)
0812 05:25:38.873165 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.873168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.873229 (+ 61us) server_negotiation.cc:407] Connection header received
0812 05:25:38.873297 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.873297 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.873309 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.873355 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:38.873477 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.873975 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.874842 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.875048 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.875121 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.875292 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:38.875327 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:38.875372 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.875427 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.875451 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:25:38.875560 938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.873016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:44061 (local address 127.24.66.67:42649)
0812 05:25:38.873175 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.873186 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.873246 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.873367 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.873367 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.873369 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:25:38.873418 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.873419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.874067 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.874068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.874741 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.874743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.875134 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.875135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.875191 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.875205 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:25:38.875382 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:25:38.875384 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.875426 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.875480 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
I20260812 05:25:38.878199 502 raft_consensus.cc:1261] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Refusing update from remote peer 2d1992b8b0374e5a874de2c92808214c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:38.878247 649 raft_consensus.cc:1261] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Refusing update from remote peer 2d1992b8b0374e5a874de2c92808214c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:38.878473 896 consensus_queue.cc:1035] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:38.878590 883 consensus_queue.cc:1035] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:38.887972 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.884709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:52425 (local address 127.24.66.67:35143)
0812 05:25:38.884761 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.884764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.884909 (+ 145us) server_negotiation.cc:407] Connection header received
0812 05:25:38.884982 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.884983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.884992 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.885033 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:38.885186 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.885669 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.886879 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.887167 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.887243 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.887675 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:38.887712 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:38.887763 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.887863 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.887893 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:25:38.888276 943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.884644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:35143 (local address 127.24.66.65:52425)
0812 05:25:38.884849 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:38.884862 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:38.884924 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:38.885048 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:38.885049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:38.885051 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 05:25:38.885131 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.885133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.885746 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.885747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.886728 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:38.886731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.887291 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.887292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:38.887380 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.887400 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 05:25:38.887788 (+ 388us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 05:25:38.887791 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:38.887857 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 05:25:38.887914 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":127,"threads_started":1}
I20260812 05:25:38.896524 332 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:38.896656 332 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:38.897324 332 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e: Sending DeleteTablet for 3 replicas of tablet 4218e2c09f17449b92f7cf02fcb30210
I20260812 05:25:38.897609 482 tablet_service.cc:1514] Processing DeleteTablet for tablet 4218e2c09f17449b92f7cf02fcb30210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:38 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:41194
I20260812 05:25:38.897632 629 tablet_service.cc:1514] Processing DeleteTablet for tablet 4218e2c09f17449b92f7cf02fcb30210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:38 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:36798
I20260812 05:25:38.897886 947 tablet_replica.cc:330] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac: stopping tablet replica
I20260812 05:25:38.897950 947 raft_consensus.cc:2229] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:38.897989 947 raft_consensus.cc:2258] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:38.897984 776 tablet_service.cc:1514] Processing DeleteTablet for tablet 4218e2c09f17449b92f7cf02fcb30210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:38 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:60956
I20260812 05:25:38.898180 948 tablet_replica.cc:330] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c: stopping tablet replica
I20260812 05:25:38.898278 948 raft_consensus.cc:2229] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:38.898288 947 ts_tablet_manager.cc:1905] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:38.898445 948 raft_consensus.cc:2258] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:38.900220 947 ts_tablet_manager.cc:1918] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:38.900296 947 log.cc:1196] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/4218e2c09f17449b92f7cf02fcb30210
I20260812 05:25:38.900565 947 ts_tablet_manager.cc:1939] T 4218e2c09f17449b92f7cf02fcb30210 P 74645a0f996c46e4bdac38c4b43fefac: Deleting consensus metadata
I20260812 05:25:38.901017 321 catalog_manager.cc:4841] TS 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117): tablet 4218e2c09f17449b92f7cf02fcb30210 (table test-table [id=1e8454c46b0240278bcb33cf75243402]) successfully deleted
I20260812 05:25:38.901772 948 ts_tablet_manager.cc:1905] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:38.903586 948 ts_tablet_manager.cc:1918] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:38.903659 948 log.cc:1196] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/4218e2c09f17449b92f7cf02fcb30210
I20260812 05:25:38.903911 948 ts_tablet_manager.cc:1939] T 4218e2c09f17449b92f7cf02fcb30210 P 2d1992b8b0374e5a874de2c92808214c: Deleting consensus metadata
I20260812 05:25:38.904336 320 catalog_manager.cc:4841] TS 2d1992b8b0374e5a874de2c92808214c (127.24.66.67:35143): tablet 4218e2c09f17449b92f7cf02fcb30210 (table test-table [id=1e8454c46b0240278bcb33cf75243402]) successfully deleted
I20260812 05:25:38.909052 946 tablet_replica.cc:330] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592: stopping tablet replica
I20260812 05:25:38.909148 946 raft_consensus.cc:2229] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:38.909196 946 raft_consensus.cc:2258] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:38.909520 946 ts_tablet_manager.cc:1905] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:38.911391 946 ts_tablet_manager.cc:1918] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:38.911463 946 log.cc:1196] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/wal/wals/4218e2c09f17449b92f7cf02fcb30210
I20260812 05:25:38.911700 946 ts_tablet_manager.cc:1939] T 4218e2c09f17449b92f7cf02fcb30210 P a838cc64cdfc456797d7bd33be666592: Deleting consensus metadata
Aug 12 05:25:38 dist-test-slave-zpfg krb5kdc[32755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COMI20260812 05:25:38.912269 319 catalog_manager.cc:4841] TS a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061): tablet 4218e2c09f17449b92f7cf02fcb30210 (table test-table [id=1e8454c46b0240278bcb33cf75243402]) successfully deleted
:
Aug 12 05:25:38 dist-test-slave-zpfg krb5kdc[32755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:38 dist-test-slave-zpfg krb5kdc[32755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 05:25:38.929981 390 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:38.918131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41526 (local address 127.24.66.126:38153)
0812 05:25:38.918185 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.918187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.918414 (+ 227us) server_negotiation.cc:407] Connection header received
0812 05:25:38.918491 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.918492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.918509 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.918580 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:38.919421 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.919943 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.921112 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.921352 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.921476 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.929907 (+ 8431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41526: BlockingRecv error: recv got EOF from 127.0.0.1:41526 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 12 05:25:38 dist-test-slave-zpfg krb5kdc[32755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM
I20260812 05:25:38.939669 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:38.931160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41532 (local address 127.24.66.126:38153)
0812 05:25:38.931226 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:38.931228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:38.931320 (+ 92us) server_negotiation.cc:407] Connection header received
0812 05:25:38.931378 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:38.931379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:38.931392 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:38.931454 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:38.931981 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.932807 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.933684 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:38.933991 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:38.934077 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:38.936509 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:38.936511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:38.936511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:38.936520 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:38.938202 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:38.938714 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:38.938716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:38.938717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:38.938790 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:38.939052 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:38.939053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:38.939053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:38.939215 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:38.939332 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:38.939478 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 05:25:38.939566 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:25:38.941672 332 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:25:38.941874 332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:38.944332 482 tablet_service.cc:1467] Processing CreateTablet for tablet 255124297a71490196363c05723d07f6 (DEFAULT_TABLE table=test-table [id=91a4342b0300464fa0c8c8291d69b1b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:38.944326 776 tablet_service.cc:1467] Processing CreateTablet for tablet 255124297a71490196363c05723d07f6 (DEFAULT_TABLE table=test-table [id=91a4342b0300464fa0c8c8291d69b1b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:38.944379 629 tablet_service.cc:1467] Processing CreateTablet for tablet 255124297a71490196363c05723d07f6 (DEFAULT_TABLE table=test-table [id=91a4342b0300464fa0c8c8291d69b1b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:38.944494 482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 255124297a71490196363c05723d07f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.944495 776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 255124297a71490196363c05723d07f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.944502 629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 255124297a71490196363c05723d07f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:38.945888 880 tablet_bootstrap.cc:492] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac: Bootstrap starting.
I20260812 05:25:38.945899 877 tablet_bootstrap.cc:492] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592: Bootstrap starting.
I20260812 05:25:38.945926 876 tablet_bootstrap.cc:492] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c: Bootstrap starting.
I20260812 05:25:38.946269 877 tablet_bootstrap.cc:654] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.946288 876 tablet_bootstrap.cc:654] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.946302 880 tablet_bootstrap.cc:654] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:38.947216 877 tablet_bootstrap.cc:492] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592: No bootstrap required, opened a new log
I20260812 05:25:38.947249 876 tablet_bootstrap.cc:492] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c: No bootstrap required, opened a new log
I20260812 05:25:38.947278 877 ts_tablet_manager.cc:1397] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:38.947296 876 ts_tablet_manager.cc:1397] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:38.947432 876 raft_consensus.cc:348] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.947451 877 raft_consensus.cc:348] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.947530 877 raft_consensus.cc:374] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.947531 876 raft_consensus.cc:374] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.947557 876 raft_consensus.cc:729] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1992b8b0374e5a874de2c92808214c, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.947548 877 raft_consensus.cc:729] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a838cc64cdfc456797d7bd33be666592, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.947604 877 consensus_queue.cc:260] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.947623 876 consensus_queue.cc:260] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.947690 877 ts_tablet_manager.cc:1428] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:25:38.947747 876 ts_tablet_manager.cc:1428] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:38.948422 880 tablet_bootstrap.cc:492] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac: No bootstrap required, opened a new log
I20260812 05:25:38.948480 880 ts_tablet_manager.cc:1397] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 05:25:38.948652 880 raft_consensus.cc:348] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.948750 880 raft_consensus.cc:374] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:38.948774 880 raft_consensus.cc:729] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74645a0f996c46e4bdac38c4b43fefac, State: Initialized, Role: FOLLOWER
I20260812 05:25:38.948817 880 consensus_queue.cc:260] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:38.948912 880 ts_tablet_manager.cc:1428] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:39.071262 884 raft_consensus.cc:482] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:39.071383 884 raft_consensus.cc:504] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.071712 884 leader_election.cc:288] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061), 2d1992b8b0374e5a874de2c92808214c (127.24.66.67:35143)
I20260812 05:25:39.075594 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:39.072059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:43677 (local address 127.24.66.67:35143)
0812 05:25:39.072123 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:39.072125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:39.072781 (+ 656us) server_negotiation.cc:407] Connection header received
0812 05:25:39.072851 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:39.072852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:39.072865 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:39.072909 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:39.073096 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.073595 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.074847 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.075194 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.075289 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:39.075297 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:39.075491 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 05:25:39.075533 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 05:25:39.075594 927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:39.072137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:51111 (local address 127.24.66.65:44061)
0812 05:25:39.072185 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:39.072187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:39.072210 (+ 23us) server_negotiation.cc:407] Connection header received
0812 05:25:39.072310 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:39.072311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:39.072323 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:39.072391 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:25:39.072674 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.073435 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.074701 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.075176 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.075241 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:39.075249 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:39.075456 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 05:25:39.075507 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:25:39.075605 965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:39.071928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:44061 (local address 127.24.66.66:51111)
0812 05:25:39.072148 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:39.072160 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:39.072231 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:39.072423 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:39.072425 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:39.072429 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:39.072532 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:39.072535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.073528 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.073530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:39.074602 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:39.074606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.075264 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.075265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:39.075379 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:39.075383 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:39.075427 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 05:25:39.075489 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":1,"spinlock_wait_cycles":896,"thread_start_us":657,"threads_started":1}
I20260812 05:25:39.075623 966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:39.071928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:35143 (local address 127.24.66.66:43677)
0812 05:25:39.072692 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:39.072705 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:39.072776 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:39.072941 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:39.072942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:39.072945 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:25:39.073018 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:39.073020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.073705 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.073706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:39.074747 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:39.074750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.075309 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.075310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:39.075390 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:39.075394 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:39.075475 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 05:25:39.075517 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":5,"thread_start_us":118,"threads_started":1}
I20260812 05:25:39.075874 502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255124297a71490196363c05723d07f6" candidate_uuid: "74645a0f996c46e4bdac38c4b43fefac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a838cc64cdfc456797d7bd33be666592" is_pre_election: true
I20260812 05:25:39.075903 787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255124297a71490196363c05723d07f6" candidate_uuid: "74645a0f996c46e4bdac38c4b43fefac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1992b8b0374e5a874de2c92808214c" is_pre_election: true
I20260812 05:25:39.075994 787 raft_consensus.cc:2454] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74645a0f996c46e4bdac38c4b43fefac in term 0.
I20260812 05:25:39.075994 502 raft_consensus.cc:2454] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74645a0f996c46e4bdac38c4b43fefac in term 0.
I20260812 05:25:39.076205 579 leader_election.cc:302] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1992b8b0374e5a874de2c92808214c, 74645a0f996c46e4bdac38c4b43fefac; no voters:
I20260812 05:25:39.076356 884 raft_consensus.cc:2781] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:39.076400 884 raft_consensus.cc:482] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:39.076424 884 raft_consensus.cc:3037] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:39.077178 884 raft_consensus.cc:504] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.077328 884 leader_election.cc:288] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [CANDIDATE]: Term 1 election: Requested vote from peers a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061), 2d1992b8b0374e5a874de2c92808214c (127.24.66.67:35143)
I20260812 05:25:39.077509 787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255124297a71490196363c05723d07f6" candidate_uuid: "74645a0f996c46e4bdac38c4b43fefac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1992b8b0374e5a874de2c92808214c"
I20260812 05:25:39.077493 502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255124297a71490196363c05723d07f6" candidate_uuid: "74645a0f996c46e4bdac38c4b43fefac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a838cc64cdfc456797d7bd33be666592"
I20260812 05:25:39.077591 502 raft_consensus.cc:3037] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:39.077591 787 raft_consensus.cc:3037] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:39.078487 787 raft_consensus.cc:2454] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74645a0f996c46e4bdac38c4b43fefac in term 1.
I20260812 05:25:39.078487 502 raft_consensus.cc:2454] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74645a0f996c46e4bdac38c4b43fefac in term 1.
I20260812 05:25:39.078696 578 leader_election.cc:302] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74645a0f996c46e4bdac38c4b43fefac, a838cc64cdfc456797d7bd33be666592; no voters:
I20260812 05:25:39.078817 884 raft_consensus.cc:2781] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:39.078938 884 raft_consensus.cc:686] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 1 LEADER]: Becoming Leader. State: Replica: 74645a0f996c46e4bdac38c4b43fefac, State: Running, Role: LEADER
I20260812 05:25:39.079023 884 consensus_queue.cc:237] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.079872 333 catalog_manager.cc:5495] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac reported cstate change: term changed from 0 to 1, leader changed from <none> to 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66). New cstate: current_term: 1 leader_uuid: "74645a0f996c46e4bdac38c4b43fefac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } health_report { overall_health: UNKNOWN } } }
I20260812 05:25:39.096786 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:39.094192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36810 (local address 127.24.66.66:38117)
0812 05:25:39.094265 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:39.094267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:39.094275 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:39.094315 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:39.094316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:39.094386 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:39.094466 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:39.094654 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.095153 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.095992 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.096187 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.096241 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:39.096430 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:39.096538 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:39.096599 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:39.096682 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 05:25:39.096699 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:25:39.097970 502 raft_consensus.cc:1261] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Refusing update from remote peer 74645a0f996c46e4bdac38c4b43fefac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:39.097971 787 raft_consensus.cc:1261] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Refusing update from remote peer 74645a0f996c46e4bdac38c4b43fefac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:39.098316 884 consensus_queue.cc:1035] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [LEADER]: Connected to new peer: Peer: permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:39.098573 968 consensus_queue.cc:1035] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:39.102038 332 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:39.102141 332 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:39.102852 332 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e: Sending DeleteTablet for 3 replicas of tablet 255124297a71490196363c05723d07f6
I20260812 05:25:39.103142 776 tablet_service.cc:1514] Processing DeleteTablet for tablet 255124297a71490196363c05723d07f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:39 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:60956
I20260812 05:25:39.103169 482 tablet_service.cc:1514] Processing DeleteTablet for tablet 255124297a71490196363c05723d07f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:39 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:41194
I20260812 05:25:39.103179 629 tablet_service.cc:1514] Processing DeleteTablet for tablet 255124297a71490196363c05723d07f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:39 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:36798
I20260812 05:25:39.103272 948 tablet_replica.cc:330] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c: stopping tablet replica
I20260812 05:25:39.103361 948 raft_consensus.cc:2229] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:39.103415 948 raft_consensus.cc:2258] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:39.103502 946 tablet_replica.cc:330] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592: stopping tablet replica
I20260812 05:25:39.103554 946 raft_consensus.cc:2229] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:39.103490 332 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 05:25:39.103597 946 raft_consensus.cc:2258] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 05:25:39.103631 332 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:25:39.103674 947 tablet_replica.cc:330] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac: stopping tablet replica
I20260812 05:25:39.103725 947 raft_consensus.cc:2229] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:39.103801 947 raft_consensus.cc:2258] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:39.104887 776 tablet_service.cc:1467] Processing CreateTablet for tablet 21065779ec9241eb9f779813a678fd0c (DEFAULT_TABLE table=test-table [id=141235a8662f41f6b0fa1ad99a884285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:39.104934 482 tablet_service.cc:1467] Processing CreateTablet for tablet 21065779ec9241eb9f779813a678fd0c (DEFAULT_TABLE table=test-table [id=141235a8662f41f6b0fa1ad99a884285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:39.105026 482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21065779ec9241eb9f779813a678fd0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:39.105027 776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21065779ec9241eb9f779813a678fd0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:39.106397 877 tablet_bootstrap.cc:492] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592: Bootstrap starting.
I20260812 05:25:39.106433 876 tablet_bootstrap.cc:492] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c: Bootstrap starting.
I20260812 05:25:39.106668 877 tablet_bootstrap.cc:654] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:39.106791 876 tablet_bootstrap.cc:654] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:39.106923 629 tablet_service.cc:1467] Processing CreateTablet for tablet 21065779ec9241eb9f779813a678fd0c (DEFAULT_TABLE table=test-table [id=141235a8662f41f6b0fa1ad99a884285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:25:39.107028 629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21065779ec9241eb9f779813a678fd0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:39.107990 880 tablet_bootstrap.cc:492] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac: Bootstrap starting.
I20260812 05:25:39.108475 880 tablet_bootstrap.cc:654] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:39.108536 877 tablet_bootstrap.cc:492] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592: No bootstrap required, opened a new log
I20260812 05:25:39.108585 877 ts_tablet_manager.cc:1397] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:25:39.108721 877 raft_consensus.cc:348] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.108798 877 raft_consensus.cc:374] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:39.108817 877 raft_consensus.cc:729] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a838cc64cdfc456797d7bd33be666592, State: Initialized, Role: FOLLOWER
I20260812 05:25:39.108867 877 consensus_queue.cc:260] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.108973 877 ts_tablet_manager.cc:1428] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:39.109509 947 ts_tablet_manager.cc:1905] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:39.111179 948 ts_tablet_manager.cc:1905] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:39.111330 947 ts_tablet_manager.cc:1918] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:39.111392 947 log.cc:1196] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/255124297a71490196363c05723d07f6
I20260812 05:25:39.111436 946 ts_tablet_manager.cc:1905] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:39.111641 947 ts_tablet_manager.cc:1939] T 255124297a71490196363c05723d07f6 P 74645a0f996c46e4bdac38c4b43fefac: Deleting consensus metadata
I20260812 05:25:39.111891 876 tablet_bootstrap.cc:492] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c: No bootstrap required, opened a new log
I20260812 05:25:39.111934 876 ts_tablet_manager.cc:1397] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260812 05:25:39.112041 876 raft_consensus.cc:348] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.112104 876 raft_consensus.cc:374] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:39.112125 876 raft_consensus.cc:729] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1992b8b0374e5a874de2c92808214c, State: Initialized, Role: FOLLOWER
I20260812 05:25:39.112169 876 consensus_queue.cc:260] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.112253 876 ts_tablet_manager.cc:1428] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:39.112257 321 catalog_manager.cc:4841] TS 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117): tablet 255124297a71490196363c05723d07f6 (table test-table [id=91a4342b0300464fa0c8c8291d69b1b3]) successfully deleted
I20260812 05:25:39.112805 880 tablet_bootstrap.cc:492] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac: No bootstrap required, opened a new log
I20260812 05:25:39.112876 880 ts_tablet_manager.cc:1397] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 05:25:39.113029 880 raft_consensus.cc:348] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.113106 880 raft_consensus.cc:374] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:39.113125 880 raft_consensus.cc:729] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74645a0f996c46e4bdac38c4b43fefac, State: Initialized, Role: FOLLOWER
I20260812 05:25:39.113178 880 consensus_queue.cc:260] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.113265 880 ts_tablet_manager.cc:1428] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 05:25:39.113626 946 ts_tablet_manager.cc:1918] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:39.113688 946 log.cc:1196] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-0/wal/wals/255124297a71490196363c05723d07f6
I20260812 05:25:39.113935 946 ts_tablet_manager.cc:1939] T 255124297a71490196363c05723d07f6 P a838cc64cdfc456797d7bd33be666592: Deleting consensus metadata
I20260812 05:25:39.114184 948 ts_tablet_manager.cc:1918] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:25:39.114243 948 log.cc:1196] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/255124297a71490196363c05723d07f6
I20260812 05:25:39.114491 948 ts_tablet_manager.cc:1939] T 255124297a71490196363c05723d07f6 P 2d1992b8b0374e5a874de2c92808214c: Deleting consensus metadata
I20260812 05:25:39.114490 319 catalog_manager.cc:4841] TS a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061): tablet 255124297a71490196363c05723d07f6 (table test-table [id=91a4342b0300464fa0c8c8291d69b1b3]) successfully deleted
I20260812 05:25:39.115127 320 catalog_manager.cc:4841] TS 2d1992b8b0374e5a874de2c92808214c (127.24.66.67:35143): tablet 255124297a71490196363c05723d07f6 (table test-table [id=91a4342b0300464fa0c8c8291d69b1b3]) successfully deleted
I20260812 05:25:39.428385 944 raft_consensus.cc:482] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:25:39.428511 944 raft_consensus.cc:504] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.428725 944 leader_election.cc:288] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061), 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117)
I20260812 05:25:39.429050 648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21065779ec9241eb9f779813a678fd0c" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74645a0f996c46e4bdac38c4b43fefac" is_pre_election: true
I20260812 05:25:39.429069 502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21065779ec9241eb9f779813a678fd0c" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a838cc64cdfc456797d7bd33be666592" is_pre_election: true
I20260812 05:25:39.429180 502 raft_consensus.cc:2454] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 0.
I20260812 05:25:39.429179 648 raft_consensus.cc:2454] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 0.
I20260812 05:25:39.429390 726 leader_election.cc:302] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1992b8b0374e5a874de2c92808214c, a838cc64cdfc456797d7bd33be666592; no voters:
I20260812 05:25:39.429509 944 raft_consensus.cc:2781] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:25:39.429563 944 raft_consensus.cc:482] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:25:39.429580 944 raft_consensus.cc:3037] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:39.430569 944 raft_consensus.cc:504] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.430727 944 leader_election.cc:288] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 election: Requested vote from peers a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061), 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117)
I20260812 05:25:39.430915 648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21065779ec9241eb9f779813a678fd0c" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74645a0f996c46e4bdac38c4b43fefac"
I20260812 05:25:39.430924 502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21065779ec9241eb9f779813a678fd0c" candidate_uuid: "2d1992b8b0374e5a874de2c92808214c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a838cc64cdfc456797d7bd33be666592"
I20260812 05:25:39.430995 648 raft_consensus.cc:3037] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:39.431021 502 raft_consensus.cc:3037] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:39.431730 502 raft_consensus.cc:2454] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 1.
I20260812 05:25:39.431778 648 raft_consensus.cc:2454] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1992b8b0374e5a874de2c92808214c in term 1.
I20260812 05:25:39.431932 726 leader_election.cc:302] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1992b8b0374e5a874de2c92808214c, a838cc64cdfc456797d7bd33be666592; no voters:
I20260812 05:25:39.432030 944 raft_consensus.cc:2781] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:39.432132 944 raft_consensus.cc:686] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 1 LEADER]: Becoming Leader. State: Replica: 2d1992b8b0374e5a874de2c92808214c, State: Running, Role: LEADER
I20260812 05:25:39.432176 944 consensus_queue.cc:237] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } }
I20260812 05:25:39.432852 332 catalog_manager.cc:5495] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d1992b8b0374e5a874de2c92808214c (127.24.66.67). New cstate: current_term: 1 leader_uuid: "2d1992b8b0374e5a874de2c92808214c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d1992b8b0374e5a874de2c92808214c" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35143 } health_report { overall_health: HEALTHY } } }
I20260812 05:25:39.469473 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:39.467123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60972 (local address 127.24.66.67:35143)
0812 05:25:39.467200 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:39.467202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:39.467212 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:39.467266 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:39.467267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:39.467278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:39.467330 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:25:39.467440 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.467944 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.468729 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:39.468950 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:39.469031 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:39.469190 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:25:39.469227 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:25:39.469300 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:39.469356 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 05:25:39.469382 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 05:25:39.472438 502 raft_consensus.cc:1261] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Refusing update from remote peer 2d1992b8b0374e5a874de2c92808214c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:39.472463 648 raft_consensus.cc:1261] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Refusing update from remote peer 2d1992b8b0374e5a874de2c92808214c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:25:39.472724 979 consensus_queue.cc:1035] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74645a0f996c46e4bdac38c4b43fefac" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:39.472841 944 consensus_queue.cc:1035] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a838cc64cdfc456797d7bd33be666592" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:25:39.484311 332 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:39.484428 332 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:25:39.485278 332 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a832ab86cff0417ba28fdaeea31d137e: Sending DeleteTablet for 3 replicas of tablet 21065779ec9241eb9f779813a678fd0c
I20260812 05:25:39.485594 629 tablet_service.cc:1514] Processing DeleteTablet for tablet 21065779ec9241eb9f779813a678fd0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:39 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:36798
I20260812 05:25:39.485638 482 tablet_service.cc:1514] Processing DeleteTablet for tablet 21065779ec9241eb9f779813a678fd0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:39 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:41194
I20260812 05:25:39.485772 946 tablet_replica.cc:330] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592: stopping tablet replica
I20260812 05:25:39.485798 947 tablet_replica.cc:330] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac: stopping tablet replica
I20260812 05:25:39.485852 946 raft_consensus.cc:2229] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:39.485850 947 raft_consensus.cc:2229] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:25:39.485893 947 raft_consensus.cc:2258] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:39.485893 946 raft_consensus.cc:2258] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:39.486238 946 ts_tablet_manager.cc:1905] T 21065779ec9241eb9f779813a678fd0c P a838cc64cdfc456797d7bd33be666592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:39.486284 947 ts_tablet_manager.cc:1905] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:39.486281 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 414
I20260812 05:25:39.488415 947 ts_tablet_manager.cc:1918] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:39.488493 947 log.cc:1196] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/21065779ec9241eb9f779813a678fd0c
I20260812 05:25:39.488757 947 ts_tablet_manager.cc:1939] T 21065779ec9241eb9f779813a678fd0c P 74645a0f996c46e4bdac38c4b43fefac: Deleting consensus metadata
I20260812 05:25:39.489315 321 catalog_manager.cc:4841] TS 74645a0f996c46e4bdac38c4b43fefac (127.24.66.66:38117): tablet 21065779ec9241eb9f779813a678fd0c (table test-table [id=141235a8662f41f6b0fa1ad99a884285]) successfully deleted
I20260812 05:25:39.491101 776 tablet_service.cc:1514] Processing DeleteTablet for tablet 21065779ec9241eb9f779813a678fd0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:25:39 UTC) from {username='oryx', principal='oryx/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:60956
I20260812 05:25:39.491326 948 tablet_replica.cc:330] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c: stopping tablet replica
I20260812 05:25:39.491432 948 raft_consensus.cc:2229] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:25:39.491531 948 raft_consensus.cc:2258] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:25:39.492003 948 ts_tablet_manager.cc:1905] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:25:39.494037 948 ts_tablet_manager.cc:1918] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 05:25:39.494112 948 log.cc:1196] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/21065779ec9241eb9f779813a678fd0c
I20260812 05:25:39.494381 948 ts_tablet_manager.cc:1939] T 21065779ec9241eb9f779813a678fd0c P 2d1992b8b0374e5a874de2c92808214c: Deleting consensus metadata
I20260812 05:25:39.494954 320 catalog_manager.cc:4841] TS 2d1992b8b0374e5a874de2c92808214c (127.24.66.67:35143): tablet 21065779ec9241eb9f779813a678fd0c (table test-table [id=141235a8662f41f6b0fa1ad99a884285]) successfully deleted
W20260812 05:25:39.498107 319 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:44061 (error 108)
I20260812 05:25:39.498456 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 561
I20260812 05:25:39.502574 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:39.498614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:44061 (local address 127.0.0.1:41226)
0812 05:25:39.502436 (+ 3822us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:39.502510 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:44061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3781,"thread_start_us":45,"threads_started":1}
W20260812 05:25:39.502805 319 catalog_manager.cc:4584] TS a838cc64cdfc456797d7bd33be666592 (127.24.66.65:44061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 21065779ec9241eb9f779813a678fd0c: Network error: Client connection negotiation failed: client connection to 127.24.66.65:44061: connect: Connection refused (error 111)
I20260812 05:25:39.507776 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 709
I20260812 05:25:39.516955 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 300
2026-08-12T05:25:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3464 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:39 dist-test-slave-zpfg krb5kdc[994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:39 dist-test-slave-zpfg krb5kdc[994](info): set up 2 sockets
Aug 12 05:25:39 dist-test-slave-zpfg krb5kdc[994](info): commencing operation
krb5kdc: starting...
W20260812 05:25:40.611438 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:40 dist-test-slave-zpfg krb5kdc[994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:25:40.754181 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:43505
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:43505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:40.845376 1007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:40.845605 1007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:40.845633 1007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:40.846964 1007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:40.847018 1007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:40.847033 1007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:40.847045 1007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:40.847057 1007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:40.848614 1007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:43505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:43505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:40.848843 1007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:40.849040 1007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:40.854592 1016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:40.854583 1013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:40.854604 1007 server_base.cc:1034] running on GCE node
W20260812 05:25:40.854583 1012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:40.855129 1007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:40.855535 1007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:40.856698 1007 hybrid_clock.cc:648] HybridClock initialized: now 1786512340856669 us; error 42 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T05:25:40Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1341 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:40 dist-test-slave-zpfg krb5kdc[1022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:40 dist-test-slave-zpfg krb5kdc[1022](info): set up 2 sockets
Aug 12 05:25:40 dist-test-slave-zpfg krb5kdc[1022](info): commencing operation
krb5kdc: starting...
W20260812 05:25:42.040508 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:25:42 dist-test-slave-zpfg krb5kdc[1022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:25:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:25:42.177259 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39389
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:39389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:42.263715 1035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:42.264003 1035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:42.264051 1035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:42.265509 1035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:42.265600 1035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:42.265630 1035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:42.265654 1035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:42.265717 1035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:42.267630 1035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:39389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:42.267970 1035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:42.268251 1035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:42.274021 1042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:42.274088 1041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:42.274425 1044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:42.274554 1035 server_base.cc:1034] running on GCE node
I20260812 05:25:42.274845 1035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:42.275187 1035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:42.276396 1035 hybrid_clock.cc:648] HybridClock initialized: now 1786512342276367 us; error 44 us; skew 500 ppm
Aug 12 05:25:42 dist-test-slave-zpfg krb5kdc[1022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:42.279660 1035 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:42.280133 1035 webserver.cc:460] Webserver started at http://127.24.66.126:38423/ using document root <none> and password file <none>
I20260812 05:25:42.280371 1035 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:42.280418 1035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:42.280527 1035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:42.281726 1035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "ab8d6948d9394cb9b345636a9e14370a"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "c2d6bbbf1909e8807932af5cd45c6af2"
server_key_iv: "63b610f985a3f0c341c72a47639eebbb"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.282084 1035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "ab8d6948d9394cb9b345636a9e14370a"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "c2d6bbbf1909e8807932af5cd45c6af2"
server_key_iv: "63b610f985a3f0c341c72a47639eebbb"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.283864 1035 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 05:25:42.284719 1051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:42.285045 1035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:42.285123 1035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "ab8d6948d9394cb9b345636a9e14370a"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "c2d6bbbf1909e8807932af5cd45c6af2"
server_key_iv: "63b610f985a3f0c341c72a47639eebbb"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.285223 1035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:42.294736 1035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:42.297140 1035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:42.297302 1035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:42.301821 1035 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:39389
I20260812 05:25:42.301857 1106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:39389 every 8 connection(s)
I20260812 05:25:42.302282 1035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:42.303535 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 1035
I20260812 05:25:42.303545 1107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:42.303670 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:42.303864 24841 external_mini_cluster.cc:1428] Setting key e8fc91953323c2aa53188576fe7640d8
I20260812 05:25:42.307543 1107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a: Bootstrap starting.
I20260812 05:25:42.308382 1107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:42.308738 1107 log.cc:824] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:42.309707 1107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a: No bootstrap required, opened a new log
Aug 12 05:25:42 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:42.312172 1107 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } }
I20260812 05:25:42.312381 1107 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:42.312471 1107 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab8d6948d9394cb9b345636a9e14370a, State: Initialized, Role: FOLLOWER
I20260812 05:25:42.312631 1107 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } }
I20260812 05:25:42.312703 1107 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:42.312727 1107 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:42.312816 1107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:42.313673 1107 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } }
I20260812 05:25:42.313823 1107 leader_election.cc:302] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab8d6948d9394cb9b345636a9e14370a; no voters:
I20260812 05:25:42.314021 1107 leader_election.cc:288] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:42.314061 1112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:42.314316 1112 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 1 LEADER]: Becoming Leader. State: Replica: ab8d6948d9394cb9b345636a9e14370a, State: Running, Role: LEADER
I20260812 05:25:42.314366 1107 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:42.314555 1112 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } }
I20260812 05:25:42.314988 1114 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab8d6948d9394cb9b345636a9e14370a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } } }
I20260812 05:25:42.315104 1114 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:42.316057 1116 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab8d6948d9394cb9b345636a9e14370a. Latest consensus state: current_term: 1 leader_uuid: "ab8d6948d9394cb9b345636a9e14370a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } } }
I20260812 05:25:42.316390 1116 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:42.316272 1123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:42.316946 1110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:42.304555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50394 (local address 127.24.66.126:39389)
0812 05:25:42.304768 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:42.304779 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:42.304820 (+ 41us) server_negotiation.cc:407] Connection header received
0812 05:25:42.305452 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:42.305463 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:42.305549 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:42.305728 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:42.306395 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.307979 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.308985 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.309607 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.309742 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:42.312344 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:42.312348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:42.312348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:42.312363 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:42.314678 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:42.315102 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:42.315106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:42.315106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:42.315415 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:42.315761 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:42.315762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:42.315763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:42.316132 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:42.316296 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:42.316600 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 05:25:42.316714 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260812 05:25:42.317119 1123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:42.318881 1123 catalog_manager.cc:1347] Generated new cluster ID: 4352b9574fd6443c807f3d96f3efa54b
I20260812 05:25:42.318941 1123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:42.336963 1123 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:42.337749 1123 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:42.345592 1123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a: Generated new TSK 0
I20260812 05:25:42.345837 1123 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:25:42.383065 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39389
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:42.482842 1137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:42.483191 1137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:42.483253 1137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:42.484697 1137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:42.484783 1137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:42.484843 1137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:42.487064 1137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:42.487465 1137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:42.487804 1137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:42.493876 1143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:42.493880 1142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:42.494120 1137 server_base.cc:1034] running on GCE node
W20260812 05:25:42.493917 1145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:42.494503 1137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:42.494809 1137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:42.496011 1137 hybrid_clock.cc:648] HybridClock initialized: now 1786512342495996 us; error 32 us; skew 500 ppm
Aug 12 05:25:42 dist-test-slave-zpfg krb5kdc[1022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:42.499073 1137 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:25:42.499708 1137 webserver.cc:460] Webserver started at http://127.24.66.65:39975/ using document root <none> and password file <none>
I20260812 05:25:42.500015 1137 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:42.500130 1137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:42.500304 1137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:42.501657 1137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "8bc01676b12c4256b48eaf1f18adc476"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "b0b572ee56c586f51e84edb67cfac07d"
server_key_iv: "72d64c5f6943e4d8a02ac65aaf9985ac"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.502082 1137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "8bc01676b12c4256b48eaf1f18adc476"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "b0b572ee56c586f51e84edb67cfac07d"
server_key_iv: "72d64c5f6943e4d8a02ac65aaf9985ac"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.503923 1137 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 05:25:42.505141 1153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:42.505416 1137 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:42.505573 1137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "8bc01676b12c4256b48eaf1f18adc476"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "b0b572ee56c586f51e84edb67cfac07d"
server_key_iv: "72d64c5f6943e4d8a02ac65aaf9985ac"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.505667 1137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:42.541133 1137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:42.543733 1137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:42.543946 1137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:42.544320 1137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:42.544683 1137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:42.544744 1137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:42.544788 1137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:42.544835 1137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:42.551569 1137 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:41753
I20260812 05:25:42.551728 1274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:41753 every 8 connection(s)
I20260812 05:25:42.552035 1137 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
Aug 12 05:25:42 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:42.560724 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 1137
I20260812 05:25:42.560849 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:42.561079 24841 external_mini_cluster.cc:1428] Setting key 9a9f58c47cefacdf34aec79c56d0ea57
I20260812 05:25:42.563010 1110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:42.552906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:51289 (local address 127.24.66.126:39389)
0812 05:25:42.552972 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:42.552975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:42.553306 (+ 331us) server_negotiation.cc:407] Connection header received
0812 05:25:42.553778 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:42.553780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:42.553794 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:42.553851 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:42.554945 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.555487 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.556552 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.556800 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.556881 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:42.559382 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:42.559383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:42.559384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:42.559392 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:42.561216 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:42.561784 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:42.561785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:42.561785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:42.561845 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:42.562210 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:42.562210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:42.562211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:42.562427 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:42.562490 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:42.562857 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 05:25:42.562942 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:25:42.563347 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:42.552860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:51289)
0812 05:25:42.553039 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:42.553088 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:42.553662 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:42.553938 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:42.553941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:42.554615 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:42.554832 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:42.554838 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.555595 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.555597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:42.556449 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:42.556451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.556891 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.556892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:42.557029 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:42.557672 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:42.557681 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:42.559261 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:42.561333 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:42.561344 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:42.561346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:42.561709 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:42.561940 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:42.561941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:42.561941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:42.562132 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:42.562502 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:42.562510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:42.562708 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 05:25:42.562838 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 05:25:42.563046 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260812 05:25:42.564015 1275 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39389
I20260812 05:25:42.564157 1275 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:42.564404 1275 heartbeater.cc:511] Master 127.24.66.126:39389 requested a full tablet report, sending...
I20260812 05:25:42.565089 1070 ts_manager.cc:194] Registered new tserver with Master: 8bc01676b12c4256b48eaf1f18adc476 (127.24.66.65:41753)
I20260812 05:25:42.566043 1070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:51289
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:25:42.595444 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39389
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:42.679997 1281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:42.680270 1281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:42.680321 1281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:42.681555 1281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:42.681631 1281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:42.681676 1281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:42.683228 1281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:42.683552 1281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:42.683802 1281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:25:42.689298 1281 server_base.cc:1034] running on GCE node
W20260812 05:25:42.689250 1286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:42.689273 1287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:42.689250 1289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:42.689879 1281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:42.690152 1281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:42.691322 1281 hybrid_clock.cc:648] HybridClock initialized: now 1786512342691289 us; error 45 us; skew 500 ppm
Aug 12 05:25:42 dist-test-slave-zpfg krb5kdc[1022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:42.694378 1281 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:25:42.694891 1281 webserver.cc:460] Webserver started at http://127.24.66.66:40885/ using document root <none> and password file <none>
I20260812 05:25:42.695096 1281 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:42.695139 1281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:42.695236 1281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:42.696367 1281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "8ad2fe63b090475199e39e9800dcf807"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "482a5c206cff0b1a2ff6d3b2ff6f3b35"
server_key_iv: "a17704f5e5ef412ac453f8b2a9c3b878"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.696718 1281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "8ad2fe63b090475199e39e9800dcf807"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "482a5c206cff0b1a2ff6d3b2ff6f3b35"
server_key_iv: "a17704f5e5ef412ac453f8b2a9c3b878"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.698071 1281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:25:42.698941 1296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:42.699122 1281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:25:42.699237 1281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "8ad2fe63b090475199e39e9800dcf807"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "482a5c206cff0b1a2ff6d3b2ff6f3b35"
server_key_iv: "a17704f5e5ef412ac453f8b2a9c3b878"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.699386 1281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:42.706301 1281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:42.708477 1281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:42.708653 1281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:42.708904 1281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:42.709278 1281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:42.709334 1281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:42.709379 1281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:42.709406 1281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:42.715862 1281 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:43799
I20260812 05:25:42.715910 1420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:43799 every 8 connection(s)
I20260812 05:25:42.716265 1281 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:42.721256 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 1281
I20260812 05:25:42.721406 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:42.721597 24841 external_mini_cluster.cc:1428] Setting key 6200760a46d5213005dcf998d545111f
Aug 12 05:25:42 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:42.726249 1110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:42.717013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:55647 (local address 127.24.66.126:39389)
0812 05:25:42.717088 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:42.717090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:42.717300 (+ 210us) server_negotiation.cc:407] Connection header received
0812 05:25:42.717779 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:42.717780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:42.717790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:42.717852 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:42.718799 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.719380 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.720399 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.720621 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.720689 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:42.723277 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:42.723279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:42.723279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:42.723287 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:42.724757 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:42.725212 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:42.725214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:42.725215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:42.725276 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:42.725556 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:42.725558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:42.725558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:42.725681 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:42.725754 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:42.726057 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 05:25:42.726153 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:25:42.726492 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:42.717004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:55647)
0812 05:25:42.717166 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:42.717209 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:42.717698 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:42.717917 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:42.717920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:42.718519 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:42.718714 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:42.718719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.719465 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.719466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:42.720290 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:42.720293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.720727 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.720727 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:42.720894 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:42.721487 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:42.721498 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:42.723173 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:42.724860 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:42.724868 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:42.724870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:42.725115 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:42.725363 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:42.725364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:42.725364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:42.725484 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:42.725759 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:42.725766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:42.725919 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 05:25:42.726035 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 05:25:42.726210 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260812 05:25:42.727150 1421 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39389
I20260812 05:25:42.727239 1421 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:42.727506 1421 heartbeater.cc:511] Master 127.24.66.126:39389 requested a full tablet report, sending...
I20260812 05:25:42.727929 1070 ts_manager.cc:194] Registered new tserver with Master: 8ad2fe63b090475199e39e9800dcf807 (127.24.66.66:43799)
I20260812 05:25:42.728507 1070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:55647
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:25:42.753537 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39389
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:42.842866 1427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:42.843083 1427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:42.843107 1427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:42.844237 1427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:42.844274 1427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:42.844300 1427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:42.845958 1427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:42.846232 1427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:42.846515 1427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:42.851907 1436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:42.851998 1434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:42.851909 1433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:42.851941 1427 server_base.cc:1034] running on GCE node
I20260812 05:25:42.852391 1427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:42.852648 1427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:42.853820 1427 hybrid_clock.cc:648] HybridClock initialized: now 1786512342853798 us; error 29 us; skew 500 ppm
Aug 12 05:25:42 dist-test-slave-zpfg krb5kdc[1022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:42.856554 1427 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:25:42.857075 1427 webserver.cc:460] Webserver started at http://127.24.66.67:42709/ using document root <none> and password file <none>
I20260812 05:25:42.857319 1427 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:42.857390 1427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:42.857543 1427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:42.858721 1427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "00e70c0efe05496c8b94386b0c5fcb84"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "5a69eacdd0759df96c35f88243a9e5cc"
server_key_iv: "161a8ca416fcd1cd7a65dcedf988c270"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.859169 1427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "00e70c0efe05496c8b94386b0c5fcb84"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "5a69eacdd0759df96c35f88243a9e5cc"
server_key_iv: "161a8ca416fcd1cd7a65dcedf988c270"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.860566 1427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:42.861358 1443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:42.861544 1427 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:42.861649 1427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "00e70c0efe05496c8b94386b0c5fcb84"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "5a69eacdd0759df96c35f88243a9e5cc"
server_key_iv: "161a8ca416fcd1cd7a65dcedf988c270"
server_key_version: "encryptionkey@0"
I20260812 05:25:42.861730 1427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:42.880096 1427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:42.882280 1427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:42.882546 1427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:42.882817 1427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:42.883157 1427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:42.883214 1427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:42.883261 1427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:42.883291 1427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:42.890172 1427 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:40451
I20260812 05:25:42.890223 1565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:40451 every 8 connection(s)
I20260812 05:25:42.890622 1427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
Aug 12 05:25:42 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:25:42.899864 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 1427
I20260812 05:25:42.899945 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:42.900135 24841 external_mini_cluster.cc:1428] Setting key 7043c0e7fa5fb7d3461fd2a86983cfe6
I20260812 05:25:42.900257 1110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:42.891393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:56507 (local address 127.24.66.126:39389)
0812 05:25:42.891462 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:42.891464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:42.891698 (+ 234us) server_negotiation.cc:407] Connection header received
0812 05:25:42.892253 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:42.892254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:42.892266 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:42.892340 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:42.893328 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.893821 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.894927 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.895186 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.895274 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:42.897538 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:42.897540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:42.897540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:42.897547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:42.898853 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:42.899235 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:42.899235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:42.899236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:42.899286 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:42.899532 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:42.899532 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:42.899533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:42.899668 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:42.899742 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:42.900028 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 05:25:42.900112 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:25:42.900416 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:42.891400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:56507)
0812 05:25:42.891576 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:42.891616 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:42.892152 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:42.892414 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:42.892418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:42.893023 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:42.893225 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:42.893230 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.893905 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.893906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:42.894831 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:42.894834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:42.895284 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:42.895285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:42.895410 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:42.896003 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:42.896012 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:42.897447 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:42.898927 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:42.898934 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:42.898936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:42.899169 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:42.899373 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:25:42.899373 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:25:42.899374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:25:42.899475 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:25:42.899735 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:42.899739 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:42.899864 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 05:25:42.899990 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 05:25:42.900145 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260812 05:25:42.901021 1566 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39389
I20260812 05:25:42.901145 1566 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:42.901346 1566 heartbeater.cc:511] Master 127.24.66.126:39389 requested a full tablet report, sending...
I20260812 05:25:42.901749 1070 ts_manager.cc:194] Registered new tserver with Master: 00e70c0efe05496c8b94386b0c5fcb84 (127.24.66.67:40451)
I20260812 05:25:42.902177 1070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:56507
I20260812 05:25:42.912285 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab.
Entry for principal oryx/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab.
WARNING: no policy specified for oryx/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab.
Entry for principal oryx/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab.
WARNING: no policy specified for oryx/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab.
Entry for principal oryx/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab.
I20260812 05:25:42.998217 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 1137
I20260812 05:25:43.004357 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 1281
I20260812 05:25:43.010113 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 1427
I20260812 05:25:43.017163 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 1035
I20260812 05:25:43.024194 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39389
--webserver_interface=127.24.66.126
--webserver_port=38423
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:39389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:25:43.108637 1575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:43.108932 1575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:43.108980 1575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:43.110224 1575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:43.110313 1575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:43.110361 1575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:43.110381 1575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:43.110405 1575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:43.111999 1575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:39389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=38423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:43.112310 1575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:43.112560 1575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:25:43.117903 1575 server_base.cc:1034] running on GCE node
W20260812 05:25:43.117925 1581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:43.117874 1583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:43.117874 1580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:43.118496 1575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:43.118825 1575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:43.120012 1575 hybrid_clock.cc:648] HybridClock initialized: now 1786512343119992 us; error 48 us; skew 500 ppm
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512343, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:43.122799 1575 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:25:43.123308 1575 webserver.cc:460] Webserver started at http://127.24.66.126:38423/ using document root <none> and password file <none>
I20260812 05:25:43.123565 1575 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:43.123657 1575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:43.125111 1575 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:43.125906 1590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:43.126119 1575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:43.126233 1575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "ab8d6948d9394cb9b345636a9e14370a"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "c2d6bbbf1909e8807932af5cd45c6af2"
server_key_iv: "63b610f985a3f0c341c72a47639eebbb"
server_key_version: "encryptionkey@0"
I20260812 05:25:43.126655 1575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:43.137373 1575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:43.139823 1575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:43.140031 1575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:43.144546 1575 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:39389
I20260812 05:25:43.144595 1647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:39389 every 8 connection(s)
I20260812 05:25:43.145002 1575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:43.146605 1648 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:25:43.147222 1648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a: Bootstrap starting.
I20260812 05:25:43.149716 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 1575
I20260812 05:25:43.150090 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:41753
--local_ip_for_outbound_sockets=127.24.66.65
--tserver_master_addrs=127.24.66.126:39389
--webserver_port=39975
--webserver_interface=127.24.66.65
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.66.65
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:25:43.157195 1648 log.cc:824] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:43.159128 1648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:25:43.159479 1648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a: Bootstrap complete.
I20260812 05:25:43.161926 1648 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } }
I20260812 05:25:43.162175 1648 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab8d6948d9394cb9b345636a9e14370a, State: Initialized, Role: FOLLOWER
I20260812 05:25:43.162387 1648 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } }
I20260812 05:25:43.162477 1648 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:43.162534 1648 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:43.162588 1648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:25:43.163707 1648 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } }
I20260812 05:25:43.163812 1648 leader_election.cc:302] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab8d6948d9394cb9b345636a9e14370a; no voters:
I20260812 05:25:43.164036 1648 leader_election.cc:288] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 05:25:43.164119 1652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 2 FOLLOWER]: Leader election won for term 2
I20260812 05:25:43.164309 1652 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [term 2 LEADER]: Becoming Leader. State: Replica: ab8d6948d9394cb9b345636a9e14370a, State: Running, Role: LEADER
I20260812 05:25:43.164366 1648 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:43.164498 1652 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } }
I20260812 05:25:43.164976 1653 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ab8d6948d9394cb9b345636a9e14370a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } } }
I20260812 05:25:43.165055 1653 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:43.165376 1654 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab8d6948d9394cb9b345636a9e14370a. Latest consensus state: current_term: 2 leader_uuid: "ab8d6948d9394cb9b345636a9e14370a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8d6948d9394cb9b345636a9e14370a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39389 } } }
I20260812 05:25:43.165449 1654 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:43.165496 1661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:43.166088 1661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:43.166232 1661 catalog_manager.cc:1259] Loaded cluster ID: 4352b9574fd6443c807f3d96f3efa54b
I20260812 05:25:43.166320 1661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:43.167905 1661 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:43.168103 1661 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ab8d6948d9394cb9b345636a9e14370a: Loaded TSK: 0
I20260812 05:25:43.168323 1661 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 05:25:43.238636 1650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:43.238854 1650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:43.238883 1650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:43.240157 1650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:43.240208 1650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:43.240237 1650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:43.241808 1650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.65.keytab
--principal=oryx/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:41753
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=39975
--tserver_master_addrs=127.24.66.126:39389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:43.242072 1650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:43.242374 1650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:43.247586 1677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:43.247603 1679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:43.247596 1676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:43.248049 1650 server_base.cc:1034] running on GCE node
I20260812 05:25:43.248296 1650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:43.248611 1650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:43.249815 1650 hybrid_clock.cc:648] HybridClock initialized: now 1786512343249787 us; error 40 us; skew 500 ppm
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512343, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:43.253365 1650 init.cc:377] Logged in from keytab as oryx/127.24.66.65@KRBTEST.COM (short username oryx)
I20260812 05:25:43.253911 1650 webserver.cc:460] Webserver started at http://127.24.66.65:39975/ using document root <none> and password file <none>
I20260812 05:25:43.254184 1650 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:43.254280 1650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:43.255817 1650 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:43.256549 1687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:43.256747 1650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:43.256845 1650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "8bc01676b12c4256b48eaf1f18adc476"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "b0b572ee56c586f51e84edb67cfac07d"
server_key_iv: "72d64c5f6943e4d8a02ac65aaf9985ac"
server_key_version: "encryptionkey@0"
I20260812 05:25:43.257220 1650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:43.265816 1650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:43.268342 1650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:43.268518 1650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:43.268788 1650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:43.269177 1650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:43.269235 1650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:43.269279 1650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:43.269305 1650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:43.276116 1650 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:41753
I20260812 05:25:43.276160 1810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:41753 every 8 connection(s)
I20260812 05:25:43.276532 1650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:43.276921 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 1650
I20260812 05:25:43.277262 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:43799
--local_ip_for_outbound_sockets=127.24.66.66
--tserver_master_addrs=127.24.66.126:39389
--webserver_port=40885
--webserver_interface=127.24.66.66
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.66.66
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:43.292255 1813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:43.277308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:50015)
0812 05:25:43.283701 (+ 6393us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:43.283749 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:43.284289 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:43.285086 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:43.285090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:43.285734 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:43.285945 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.285951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.287239 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.287240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.288098 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.288100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.288746 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.288747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.288896 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.289503 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:43.289512 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:43.292011 (+ 2499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":6339,"thread_start_us":93,"threads_started":1}
W20260812 05:25:43.292578 1811 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:39389 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.66.126:39389: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
W20260812 05:25:43.293241 1814 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:43.277355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:50015 (local address 127.24.66.126:39389)
0812 05:25:43.284493 (+ 7138us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:43.284501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:43.284690 (+ 189us) server_negotiation.cc:407] Connection header received
0812 05:25:43.284825 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:43.284831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:43.284875 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:43.285024 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:43.286095 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.287105 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.288210 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.288638 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.288777 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.292158 (+ 3381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:50015: BlockingRecv error: recv got EOF from 127.24.66.65:50015 (error 108)
Metrics: {"server-negotiator.queue_time_us":7079,"thread_start_us":82,"threads_started":1}
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:43.298092 1813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:43.292877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:46925)
0812 05:25:43.293406 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:43.293418 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:43.293470 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:43.293646 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:43.293646 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:43.293809 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:43.293861 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.293863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.294671 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.294672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.295179 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.295181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.295578 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.295579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.295652 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.296159 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:43.296165 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:43.297995 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509}
W20260812 05:25:43.298199 1816 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:43.292939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:46925 (local address 127.24.66.126:39389)
0812 05:25:43.293132 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:43.293135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:43.293521 (+ 386us) server_negotiation.cc:407] Connection header received
0812 05:25:43.293554 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:43.293554 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:43.293564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:43.293642 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:43.293954 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.294591 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.295282 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.295502 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.295580 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.298112 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:46925: BlockingRecv error: recv got EOF from 127.24.66.65:46925 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":128,"threads_started":1}
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:43.303431 1813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:43.298760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:50839)
0812 05:25:43.298862 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:43.298873 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:43.298920 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:43.299069 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:43.299070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:43.299209 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:43.299242 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.299243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.299892 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.299893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.300397 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.300399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.300753 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.300753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.300807 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.301271 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:43.301277 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:43.303339 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87}
W20260812 05:25:43.303730 1816 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:43.298824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:50839 (local address 127.24.66.126:39389)
0812 05:25:43.298864 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:43.298866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:43.298904 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:25:43.298980 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:43.298980 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:43.298990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:43.299066 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:43.299334 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.299808 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.300482 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.300679 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.300755 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.303670 (+ 2915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:50839: BlockingRecv error: recv got EOF from 127.24.66.65:50839 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 05:25:43.303763 1811 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:25:43.374445 1815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:43.374733 1815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:43.374764 1815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:43.376144 1815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:43.376190 1815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:43.376221 1815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:43.377820 1815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.66.keytab
--principal=oryx/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:43799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=40885
--tserver_master_addrs=127.24.66.126:39389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:43.378038 1815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:43.378264 1815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:43.384181 1823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:43.384192 1824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:43.384306 1826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:43.384580 1815 server_base.cc:1034] running on GCE node
I20260812 05:25:43.384829 1815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:43.385082 1815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:43.386260 1815 hybrid_clock.cc:648] HybridClock initialized: now 1786512343386245 us; error 31 us; skew 500 ppm
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512343, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:43.389079 1815 init.cc:377] Logged in from keytab as oryx/127.24.66.66@KRBTEST.COM (short username oryx)
I20260812 05:25:43.389554 1815 webserver.cc:460] Webserver started at http://127.24.66.66:40885/ using document root <none> and password file <none>
I20260812 05:25:43.389803 1815 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:43.389869 1815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:43.391557 1815 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:43.392371 1833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:43.392618 1815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:43.392740 1815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "8ad2fe63b090475199e39e9800dcf807"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "482a5c206cff0b1a2ff6d3b2ff6f3b35"
server_key_iv: "a17704f5e5ef412ac453f8b2a9c3b878"
server_key_version: "encryptionkey@0"
I20260812 05:25:43.393114 1815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:43.428726 1815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:43.431545 1815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:43.431754 1815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:43.432042 1815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:43.432408 1815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:43.432474 1815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:43.432523 1815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:43.432554 1815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:43.439695 1815 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:43799
I20260812 05:25:43.439749 1960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:43799 every 8 connection(s)
I20260812 05:25:43.440181 1815 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:43.444445 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 1815
I20260812 05:25:43.444849 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:40451
--local_ip_for_outbound_sockets=127.24.66.67
--tserver_master_addrs=127.24.66.126:39389
--webserver_port=42709
--webserver_interface=127.24.66.67
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.66.67
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:43.448829 1964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:43.441052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:53781)
0812 05:25:43.441219 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:43.441259 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:43.441795 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:43.442113 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:43.442117 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:43.442828 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:43.443037 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.443042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.443788 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.443789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.444716 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.444718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.445236 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.445237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.445357 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.445945 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:43.445954 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:43.448657 (+ 2703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
W20260812 05:25:43.448890 1816 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:43.441091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:53781 (local address 127.24.66.126:39389)
0812 05:25:43.441156 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:43.441159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:43.441376 (+ 217us) server_negotiation.cc:407] Connection header received
0812 05:25:43.441951 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:43.441952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:43.441968 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:43.442027 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:43.443144 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.443681 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.444808 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.445096 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.445163 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.448789 (+ 3626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:53781: BlockingRecv error: recv got EOF from 127.24.66.66:53781 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260812 05:25:43.449328 1961 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:39389 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.66.126:39389: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:43.454756 1964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:43.449579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:50875)
0812 05:25:43.449636 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:43.449648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:43.449723 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:43.449879 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:43.449879 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:43.450035 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:43.450089 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.450091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.450882 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.450883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.451453 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.451455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.451870 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.451871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.451953 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.452481 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:43.452486 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:43.454651 (+ 2165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 05:25:43.454833 1816 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:43.449651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:50875 (local address 127.24.66.126:39389)
0812 05:25:43.449733 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:43.449735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:43.449749 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:25:43.449781 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:43.449782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:43.449796 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:43.449876 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:43.450176 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.450781 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.451562 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.451789 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.451873 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.454761 (+ 2888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:50875: BlockingRecv error: recv got EOF from 127.24.66.66:50875 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:43.460109 1964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:43.455167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:43171)
0812 05:25:43.455227 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:43.455241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:43.455307 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:43.455455 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:43.455456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:43.455597 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:43.455630 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.455631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.456350 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.456351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.456903 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.456905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.457310 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.457311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.457370 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.457873 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:43.457878 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:43.460029 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 05:25:43.460251 1816 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:43.455247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:43171 (local address 127.24.66.126:39389)
0812 05:25:43.455318 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:43.455321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:43.455333 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:25:43.455361 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:43.455363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:43.455373 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:43.455451 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:43.455716 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.456260 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.457000 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.457218 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.457297 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.460142 (+ 2845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:43171: BlockingRecv error: recv got EOF from 127.24.66.66:43171 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 05:25:43.460457 1961 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:25:43.533598 1965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:43.533864 1965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:43.533890 1965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:43.535183 1965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:43.535234 1965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:43.535262 1965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:43.536769 1965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/krb5kdc/oryx_127.24.66.67.keytab
--principal=oryx/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:40451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=42709
--tserver_master_addrs=127.24.66.126:39389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:43.536993 1965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:43.537186 1965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:43.542557 1974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:43.542563 1970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:43.542615 1972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:43.542800 1965 server_base.cc:1034] running on GCE node
I20260812 05:25:43.543162 1965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:43.543444 1965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:43.544620 1965 hybrid_clock.cc:648] HybridClock initialized: now 1786512343544608 us; error 54 us; skew 500 ppm
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512343, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:25:43.547554 1965 init.cc:377] Logged in from keytab as oryx/127.24.66.67@KRBTEST.COM (short username oryx)
I20260812 05:25:43.548133 1965 webserver.cc:460] Webserver started at http://127.24.66.67:42709/ using document root <none> and password file <none>
I20260812 05:25:43.548403 1965 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:43.548501 1965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:43.549995 1965 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:25:43.550743 1981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:43.550990 1965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:43.551101 1965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "00e70c0efe05496c8b94386b0c5fcb84"
format_stamp: "Formatted at 2026-08-12 05:25:42 on dist-test-slave-zpfg"
server_key: "5a69eacdd0759df96c35f88243a9e5cc"
server_key_iv: "161a8ca416fcd1cd7a65dcedf988c270"
server_key_version: "encryptionkey@0"
I20260812 05:25:43.551472 1965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:43.565485 1965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:43.568141 1965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:43.568356 1965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:43.568627 1965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:43.568975 1965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:43.569031 1965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:43.569095 1965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:43.569128 1965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:43.575884 1965 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:40451
I20260812 05:25:43.575947 2107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:40451 every 8 connection(s)
I20260812 05:25:43.576356 1965 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:43.582401 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 1965
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 05:25:43.584860 1816 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:43.577212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:40631 (local address 127.24.66.126:39389)
0812 05:25:43.577325 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:43.577327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:43.577547 (+ 220us) server_negotiation.cc:407] Connection header received
0812 05:25:43.578052 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:43.578054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:43.578065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:43.578144 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:43.579252 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.579776 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.580894 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.581097 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.581267 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.584753 (+ 3486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:40631: BlockingRecv error: recv got EOF from 127.24.66.67:40631 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 05:25:43.584903 2110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:43.577181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:40631)
0812 05:25:43.577403 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:43.577448 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:43.577942 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:43.578203 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:43.578208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:43.578898 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:43.579114 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.579120 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.579886 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.579886 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.580792 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.580795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.581287 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.581288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.581418 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.582017 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:43.582025 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:43.584645 (+ 2620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":97,"threads_started":1}
W20260812 05:25:43.585179 2108 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:39389 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.66.126:39389: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:43.590441 2110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:43.585450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:39785)
0812 05:25:43.585482 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:43.585493 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:43.585532 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:43.585648 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:43.585648 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:43.585834 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:43.585885 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.585887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.586696 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.586701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.587185 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.587187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.587634 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.587635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.587702 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.588244 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:43.588249 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:43.590322 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":14}
W20260812 05:25:43.590552 1816 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:43.585504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:39785 (local address 127.24.66.126:39389)
0812 05:25:43.585556 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:43.585558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:43.585567 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:43.585589 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:43.585590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:43.585598 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:43.585647 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:43.585962 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.586601 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.587269 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.587542 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.587625 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.590473 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:39785: BlockingRecv error: recv got EOF from 127.24.66.67:39785 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:25:43.591444 1814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:43.583006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50410 (local address 127.24.66.126:39389)
0812 05:25:43.583060 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:43.583062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:43.583073 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:43.583149 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:43.583150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:43.583166 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:43.583230 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:43.583963 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.584717 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.585790 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.586067 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.586117 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.587472 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:43.587476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:43.587476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:25:43.587488 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:43.589581 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:43.590009 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:43.590014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:43.590015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:43.590160 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:25:43.590522 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:25:43.590524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:25:43.590524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:25:43.590892 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:43.591048 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:43.591191 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 05:25:43.591310 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:43 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:43.596323 2110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:43.590831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:47437)
0812 05:25:43.590944 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:43.590956 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:43.590996 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:43.591128 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:43.591129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:43.591276 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:43.591315 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.591317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.592475 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.592477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.593108 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:43.593110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.593539 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.593540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:43.593584 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.594067 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:43.594073 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:43.596267 (+ 2194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89}
W20260812 05:25:43.596513 2108 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:25:43.596519 1816 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:43.590950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:47437 (local address 127.24.66.126:39389)
0812 05:25:43.590991 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:43.590993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:43.591001 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:43.591062 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:43.591062 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:43.591073 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:43.591126 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:43.591441 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.592370 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.593248 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:43.593457 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:43.593527 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:43.596446 (+ 2919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:47437: BlockingRecv error: recv got EOF from 127.24.66.67:47437 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 05:25:44 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:44 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:44.315274 2112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:44.304374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:58845)
0812 05:25:44.304529 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:44.304547 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:44.304606 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:44.304792 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:44.304793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:44.305002 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:44.305058 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:44.305060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.305889 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.305890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:44.306573 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:44.306576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.306984 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.306985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:44.307051 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:44.313025 (+ 5974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:44.313032 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:44.315169 (+ 2137us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
W20260812 05:25:44.315359 2113 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:44.304464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:58845 (local address 127.24.66.126:39389)
0812 05:25:44.304614 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:44.304616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:44.304625 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:44.304654 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:44.304655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:44.304664 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:44.304741 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:44.305169 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.305801 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.306694 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.306904 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.306985 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:44.315289 (+ 8304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:58845: BlockingRecv error: recv got EOF from 127.24.66.65:58845 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
Aug 12 05:25:44 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:44 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:44.466914 2114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:44.461024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:37647)
0812 05:25:44.461192 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:44.461202 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:44.461269 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:44.461506 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:44.461507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:44.461694 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:44.461753 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:44.461755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.462665 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.462666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:44.463243 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:44.463245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.463664 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.463665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:44.463723 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:44.464396 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:44.464402 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:44.466844 (+ 2442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
W20260812 05:25:44.467088 2113 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:44.461091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:37647 (local address 127.24.66.126:39389)
0812 05:25:44.461143 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:44.461144 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:44.461350 (+ 206us) server_negotiation.cc:407] Connection header received
0812 05:25:44.461377 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:44.461378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:44.461391 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:44.461438 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:44.461840 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.462552 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.463348 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.463564 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.463629 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:44.466991 (+ 3362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:37647: BlockingRecv error: recv got EOF from 127.24.66.66:37647 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 05:25:44 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:44 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:44.601754 2115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:44.596922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:55791)
0812 05:25:44.597089 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:44.597100 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:44.597161 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:44.597278 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:44.597278 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:44.597511 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:44.597563 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:44.597565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.598249 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.598250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:44.598838 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:44.598841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.599168 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.599169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:44.599214 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:44.599749 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:44.599754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:44.601686 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
W20260812 05:25:44.601907 2113 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:44.596967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:55791 (local address 127.24.66.126:39389)
0812 05:25:44.597036 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:44.597039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:44.597151 (+ 112us) server_negotiation.cc:407] Connection header received
0812 05:25:44.597204 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:44.597205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:44.597220 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:44.597273 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:44.597637 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.598177 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.598914 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:44.599097 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:44.599150 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:44.601789 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:55791: BlockingRecv error: recv got EOF from 127.24.66.67:55791 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 05:25:45 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:45 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:45.321167 2116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:45.315883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:49419)
0812 05:25:45.316053 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:45.316064 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:45.316125 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:45.316341 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:45.316343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:45.316553 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:45.316618 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:45.316619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.317301 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.317302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:45.317855 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:45.317857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.318456 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.318457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:45.318523 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:45.319071 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:45.319079 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:45.321103 (+ 2024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
W20260812 05:25:45.321370 2117 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:45.315969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:49419 (local address 127.24.66.126:39389)
0812 05:25:45.316132 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:45.316133 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:45.316162 (+ 29us) server_negotiation.cc:407] Connection header received
0812 05:25:45.316183 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:45.316183 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:45.316193 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:45.316243 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:45.316698 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.317219 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.317969 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.318246 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.318319 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:45.321288 (+ 2969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:49419: BlockingRecv error: recv got EOF from 127.24.66.65:49419 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
Aug 12 05:25:45 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:45 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:45.473147 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:45.467512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:44881)
0812 05:25:45.467695 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:45.467707 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:45.467772 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:45.467956 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:45.467958 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:45.468276 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:45.468336 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:45.468339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.469077 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.469078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:45.469895 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:45.469898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.470391 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.470392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:45.470464 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:45.470994 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:45.470999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:45.473085 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
W20260812 05:25:45.473322 2117 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:45.467679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:44881 (local address 127.24.66.126:39389)
0812 05:25:45.467750 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:45.467752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:45.467764 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:25:45.467831 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:45.467832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:45.467844 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:45.467900 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:45.468454 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.468972 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.469957 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.470261 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.470379 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:45.473231 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:44881: BlockingRecv error: recv got EOF from 127.24.66.66:44881 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 05:25:45 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:45 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:45.607765 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:45.602466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:53415)
0812 05:25:45.602636 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:45.602648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:45.602700 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:45.602900 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:45.602901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:45.603067 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:45.603117 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:45.603119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.603843 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.603844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:45.604479 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:45.604482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.605024 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.605025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:45.605082 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:45.605784 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:45.605796 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:45.607703 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
W20260812 05:25:45.608001 2117 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:45.602689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:53415 (local address 127.24.66.126:39389)
0812 05:25:45.602760 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:45.602762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:45.602771 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:45.602797 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:45.602798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:45.602809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:45.602873 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:45.603216 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.603762 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.604564 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:45.604900 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:45.604971 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:45.607909 (+ 2938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:53415: BlockingRecv error: recv got EOF from 127.24.66.67:53415 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 05:25:46 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:46 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:46.326610 2120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:46.321747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:45823)
0812 05:25:46.321900 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:46.321918 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:46.321971 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:46.322115 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:46.322116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:46.322287 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:46.322388 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:46.322391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.323057 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.323058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:46.323620 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:46.323622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.324035 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.324036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:46.324080 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:46.324603 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:46.324608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:46.326535 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
W20260812 05:25:46.326849 2121 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:46.321813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:45823 (local address 127.24.66.126:39389)
0812 05:25:46.321951 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:46.321953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:46.321964 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:46.322019 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:46.322019 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:46.322031 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:46.322095 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:46.322458 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.322980 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.323724 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.323956 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.324018 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:46.326758 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:45823: BlockingRecv error: recv got EOF from 127.24.66.65:45823 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
Aug 12 05:25:46 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:46 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:46.478987 2122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:46.473769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:37515)
0812 05:25:46.474066 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:46.474077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:46.474168 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:46.474450 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:46.474451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:46.474641 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:46.474690 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:46.474691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.475383 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.475384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:46.475923 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:46.475924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.476313 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.476314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:46.476372 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:46.476904 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:46.476909 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:46.478871 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":219,"threads_started":1}
W20260812 05:25:46.479024 2121 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:46.473998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:37515 (local address 127.24.66.126:39389)
0812 05:25:46.474054 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:46.474057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:46.474156 (+ 99us) server_negotiation.cc:407] Connection header received
0812 05:25:46.474246 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:46.474247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:46.474265 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:46.474432 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:46.474776 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.475286 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.476014 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.476225 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.476319 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:46.478959 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:37515: BlockingRecv error: recv got EOF from 127.24.66.66:37515 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 12 05:25:46 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:46 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:46.613459 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:46.608242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:44751)
0812 05:25:46.608509 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:46.608522 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:46.608600 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:46.608783 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:46.608784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:46.608981 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:46.609035 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:46.609037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.609857 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.609858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:46.610445 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:46.610447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.610823 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.610824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:46.610884 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:46.611379 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:46.611384 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:46.613382 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":164,"threads_started":1}
W20260812 05:25:46.613536 2121 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:46.608369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:44751 (local address 127.24.66.126:39389)
0812 05:25:46.608496 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:46.608498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:46.608585 (+ 87us) server_negotiation.cc:407] Connection header received
0812 05:25:46.608688 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:46.608689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:46.608708 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:46.608777 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:46.609132 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.609752 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.610536 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:46.610734 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:46.610822 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:46.613486 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:44751: BlockingRecv error: recv got EOF from 127.24.66.67:44751 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 12 05:25:47 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:47 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:47.332717 2125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:47.327213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:44071)
0812 05:25:47.327422 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:47.327435 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:47.327513 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:47.327667 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:47.327668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:47.327852 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:47.327907 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:47.327909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.328617 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.328618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:47.329260 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:47.329262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.329713 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.329732 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:47.329800 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:47.330300 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:47.330304 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:47.332653 (+ 2349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":158,"threads_started":1}
W20260812 05:25:47.332860 2126 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:47.327338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:44071 (local address 127.24.66.126:39389)
0812 05:25:47.327461 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:47.327464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:47.327498 (+ 34us) server_negotiation.cc:407] Connection header received
0812 05:25:47.327570 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:47.327571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:47.327581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:47.327645 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:47.327981 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.328528 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.329378 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.329628 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.329711 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:47.332770 (+ 3059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:44071: BlockingRecv error: recv got EOF from 127.24.66.65:44071 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
Aug 12 05:25:47 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:47 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:47.484285 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:47.479547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:37861)
0812 05:25:47.479712 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:47.479727 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:47.479795 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:47.479956 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:47.479957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:47.480130 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:47.480183 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:47.480184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.480852 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.480853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:47.481379 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:47.481380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.481742 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.481743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:47.481787 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:47.482364 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:47.482369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:47.484224 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
W20260812 05:25:47.484455 2126 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:47.479645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:37861 (local address 127.24.66.126:39389)
0812 05:25:47.479720 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:47.479722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:47.479797 (+ 75us) server_negotiation.cc:407] Connection header received
0812 05:25:47.479845 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:47.479846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:47.479857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:47.479916 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:47.480256 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.480768 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.481480 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.481669 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.481727 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:47.484372 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:37861: BlockingRecv error: recv got EOF from 127.24.66.66:37861 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 05:25:47 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:47 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:47.619457 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:47.614165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:49293)
0812 05:25:47.614360 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:47.614372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:47.614458 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:47.614621 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:47.614622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:47.614802 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:47.614858 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:47.614860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.615587 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.615589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:47.616218 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:47.616221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.616621 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.616622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:47.616668 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:47.617217 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:47.617221 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:47.619357 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
W20260812 05:25:47.619645 2126 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:47.614266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:49293 (local address 127.24.66.126:39389)
0812 05:25:47.614489 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:47.614491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:47.614530 (+ 39us) server_negotiation.cc:407] Connection header received
0812 05:25:47.614557 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:47.614558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:47.614571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:47.614614 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:47.614971 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.615486 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.616337 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:47.616537 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:47.616638 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:47.619491 (+ 2853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:49293: BlockingRecv error: recv got EOF from 127.24.66.67:49293 (error 108)
Metrics: {"server-negotiator.queue_time_us":192}
Aug 12 05:25:48 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:48 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:48.338231 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:48.333170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:47277)
0812 05:25:48.333387 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:48.333400 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:48.333473 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:48.333625 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:48.333626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:48.333807 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:48.333862 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:48.333864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.334595 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.334596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:48.335216 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:48.335218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.335591 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.335592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:48.335642 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:48.336128 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:48.336134 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:48.338168 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
W20260812 05:25:48.338430 2131 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:48.333264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:47277 (local address 127.24.66.126:39389)
0812 05:25:48.333405 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:48.333407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:48.333467 (+ 60us) server_negotiation.cc:407] Connection header received
0812 05:25:48.333527 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:48.333527 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:48.333537 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:48.333597 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:48.333926 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.334486 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.335292 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.335515 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.335568 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:48.338308 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:47277: BlockingRecv error: recv got EOF from 127.24.66.65:47277 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":54,"threads_started":1}
Aug 12 05:25:48 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:48 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:48.490095 2133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:48.485008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:45459)
0812 05:25:48.485181 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:48.485196 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:48.485271 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:48.485488 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:48.485489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:48.485738 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:48.485802 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:48.485804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.486680 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.486681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:48.487219 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:48.487221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.487575 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.487575 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:48.487620 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:48.488171 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:48.488176 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:48.490034 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":91,"threads_started":1}
W20260812 05:25:48.490252 2131 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:48.485176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:45459 (local address 127.24.66.126:39389)
0812 05:25:48.485264 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:48.485266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:48.485277 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:48.485377 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:48.485378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:48.485389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:48.485474 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:48.485916 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.486603 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.487302 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.487503 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.487560 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:48.490169 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:45459: BlockingRecv error: recv got EOF from 127.24.66.66:45459 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 05:25:48 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:48 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:48.625447 2134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:48.620199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:50275)
0812 05:25:48.620369 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:48.620380 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:48.620478 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:48.620738 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:48.620739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:48.620942 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:48.620993 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:48.620995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.621710 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.621712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:48.622302 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:48.622304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.622789 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.622790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:48.622851 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:48.623432 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:48.623445 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:48.625343 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
W20260812 05:25:48.625509 2131 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:48.620255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:50275 (local address 127.24.66.126:39389)
0812 05:25:48.620308 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:48.620310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:48.620460 (+ 150us) server_negotiation.cc:407] Connection header received
0812 05:25:48.620535 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:48.620536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:48.620551 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:48.620711 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:48.621089 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.621617 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.622465 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:48.622692 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:48.622786 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:48.625442 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:50275: BlockingRecv error: recv got EOF from 127.24.66.67:50275 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 05:25:49 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:49 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:49.348160 2136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:49.338759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:60287)
0812 05:25:49.338961 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:49.338988 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:49.339064 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:49.339278 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:49.339279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:49.339452 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:49.339505 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:49.339506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.340465 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.340466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:49.341024 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:49.341026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.341463 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.341464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:49.341518 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:49.345991 (+ 4473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:49.346019 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:49.348064 (+ 2045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":126,"threads_started":1}
W20260812 05:25:49.348248 2135 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:49.338805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:60287 (local address 127.24.66.126:39389)
0812 05:25:49.338959 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:49.338961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:49.339042 (+ 81us) server_negotiation.cc:407] Connection header received
0812 05:25:49.339160 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:49.339161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:49.339177 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:49.339257 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:49.339572 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.340381 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.341100 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.341295 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.341366 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:49.348174 (+ 6808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:60287: BlockingRecv error: recv got EOF from 127.24.66.65:60287 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
Aug 12 05:25:49 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:49 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:49.496186 2137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:49.490675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:35649)
0812 05:25:49.490885 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:49.490900 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:49.490984 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:49.491147 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:49.491148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:49.491349 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:49.491398 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:49.491400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.492095 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.492096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:49.492715 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:49.492718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.493092 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.493093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:49.493152 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:49.493631 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:49.493635 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:49.496073 (+ 2438us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":90,"threads_started":1}
W20260812 05:25:49.496241 2135 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:49.490783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:35649 (local address 127.24.66.126:39389)
0812 05:25:49.490894 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:49.490897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:49.490962 (+ 65us) server_negotiation.cc:407] Connection header received
0812 05:25:49.491048 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:49.491049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:49.491063 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:49.491136 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:49.491486 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.492004 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.492819 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.493018 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.493093 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:49.496176 (+ 3083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:35649: BlockingRecv error: recv got EOF from 127.24.66.66:35649 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 12 05:25:49 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:49 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:49.631331 2138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:49.626109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:60649)
0812 05:25:49.626284 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:49.626294 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:49.626441 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:49.626647 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:49.626649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:49.626875 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:49.626927 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:49.626929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.627623 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.627624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:49.628172 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:49.628173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.628569 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.628570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:49.628609 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:49.629298 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:49.629322 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:49.631262 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":84,"threads_started":1}
W20260812 05:25:49.631513 2135 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:49.626263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:60649 (local address 127.24.66.126:39389)
0812 05:25:49.626317 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:49.626319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:49.626427 (+ 108us) server_negotiation.cc:407] Connection header received
0812 05:25:49.626507 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:49.626508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:49.626524 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:49.626621 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:49.627003 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.627553 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.628277 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:49.628493 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:49.628560 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:49.631425 (+ 2865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:60649: BlockingRecv error: recv got EOF from 127.24.66.67:60649 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 12 05:25:50 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:50 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:50.353551 2139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:50.348773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:33549)
0812 05:25:50.348947 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:50.348961 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:50.349027 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:50.349175 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:50.349175 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:50.349370 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:50.349421 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:50.349423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.350086 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.350087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:50.350727 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:50.350730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.351076 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.351077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:50.351114 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:50.351617 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:50.351622 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:50.353489 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
W20260812 05:25:50.353693 2140 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:50.348856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:33549 (local address 127.24.66.126:39389)
0812 05:25:50.349005 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:50.349007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:50.349016 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:50.349086 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:50.349086 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:50.349097 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:50.349148 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:50.349489 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.350012 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.350813 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.351012 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.351067 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:50.353598 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:33549: BlockingRecv error: recv got EOF from 127.24.66.65:33549 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
Aug 12 05:25:50 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:50 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:50.501983 2141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:50.496709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:54063)
0812 05:25:50.496888 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:50.496900 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:50.496966 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:50.497108 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:50.497109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:50.497281 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:50.497380 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:50.497382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.498101 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.498102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:50.498681 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:50.498683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.499074 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.499075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:50.499124 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:50.499669 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:50.499674 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:50.501894 (+ 2220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":121,"threads_started":1}
W20260812 05:25:50.502209 2140 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:50.496758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:54063 (local address 127.24.66.126:39389)
0812 05:25:50.496885 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:50.496888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:50.496965 (+ 77us) server_negotiation.cc:407] Connection header received
0812 05:25:50.497030 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:50.497031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:50.497045 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:50.497098 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:50.497488 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.498006 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.498770 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.498996 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.499063 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:50.502126 (+ 3063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:54063: BlockingRecv error: recv got EOF from 127.24.66.66:54063 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 12 05:25:50 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:50 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:50.637024 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:50.631876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:40681)
0812 05:25:50.632030 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:50.632043 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:50.632104 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:50.632328 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:50.632329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:50.632509 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:50.632559 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:50.632561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.633265 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.633267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:50.633831 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:50.633834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.634197 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.634197 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:50.634258 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:50.634910 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:50.634915 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:50.636964 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
W20260812 05:25:50.637174 2140 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:50.632110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:40681 (local address 127.24.66.126:39389)
0812 05:25:50.632191 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:50.632193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:50.632201 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:25:50.632225 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:50.632226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:50.632239 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:50.632297 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:50.632641 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.633168 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.633921 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:50.634118 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:50.634196 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:50.637083 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:40681: BlockingRecv error: recv got EOF from 127.24.66.67:40681 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 05:25:51 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:51 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:51.359961 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:51.354161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:43673)
0812 05:25:51.354323 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:51.354356 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:51.354426 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:51.354626 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:51.354628 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:51.354860 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:51.354910 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:51.354912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.355674 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.355675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:51.356304 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:51.356307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.356696 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.356697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:51.356756 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:51.357351 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:51.357357 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:51.359857 (+ 2500us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":71,"threads_started":1}
W20260812 05:25:51.360092 2144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:51.354275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:43673 (local address 127.24.66.126:39389)
0812 05:25:51.354419 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:51.354421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:51.354432 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:25:51.354499 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:51.354500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:51.354514 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:51.354601 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:51.354991 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.355593 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.356395 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.356611 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.356693 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:51.359991 (+ 3298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:43673: BlockingRecv error: recv got EOF from 127.24.66.65:43673 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":108,"threads_started":1}
Aug 12 05:25:51 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:51 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:51.508526 2146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:51.502711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:55171)
0812 05:25:51.502938 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:51.502951 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:51.503053 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:51.503247 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:51.503248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:51.503444 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:51.503494 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:51.503496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.504451 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.504452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:51.505042 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:51.505043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.505523 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.505524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:51.505631 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:51.506239 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:51.506244 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:51.508465 (+ 2221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":98,"threads_started":1}
W20260812 05:25:51.508684 2144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:51.502820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:55171 (local address 127.24.66.126:39389)
0812 05:25:51.502916 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:51.502919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:51.503039 (+ 120us) server_negotiation.cc:407] Connection header received
0812 05:25:51.503133 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:51.503134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:51.503150 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:51.503233 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:51.503586 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.504353 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.505144 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.505419 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.505519 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:51.508596 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:55171: BlockingRecv error: recv got EOF from 127.24.66.66:55171 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 05:25:51 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:51 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:51.642932 2147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:51.637663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:45085)
0812 05:25:51.637857 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:51.637873 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:51.637943 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:51.638130 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:51.638131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:51.638308 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:51.638446 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:51.638449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.639104 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.639105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:51.639659 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:51.639661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.640017 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.640018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:51.640059 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:51.640616 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:51.640622 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:51.642819 (+ 2197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":131,"threads_started":1}
W20260812 05:25:51.643006 2144 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:51.637754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:45085 (local address 127.24.66.126:39389)
0812 05:25:51.637829 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:51.637831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:51.637937 (+ 106us) server_negotiation.cc:407] Connection header received
0812 05:25:51.637996 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:51.637996 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:51.638010 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:51.638077 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:51.638519 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.639028 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.639737 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:51.639952 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:51.640006 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:51.642934 (+ 2928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:45085: BlockingRecv error: recv got EOF from 127.24.66.67:45085 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 05:25:52 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:52 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:52.365432 2148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:52.360485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:38629)
0812 05:25:52.360656 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:52.360666 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:52.360729 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:52.360865 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:52.360866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:52.361041 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:52.361092 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:52.361093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.361801 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.361802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:52.362389 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:52.362391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.362725 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.362726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:52.362766 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:52.363247 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:52.363252 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:52.365305 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":97,"threads_started":1}
W20260812 05:25:52.365495 2149 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:52.360543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:38629 (local address 127.24.66.126:39389)
0812 05:25:52.360672 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:52.360674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:52.360717 (+ 43us) server_negotiation.cc:407] Connection header received
0812 05:25:52.360774 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:52.360775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:52.360785 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:52.360857 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:52.361185 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.361724 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.362463 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.362660 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.362718 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:52.365417 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:38629: BlockingRecv error: recv got EOF from 127.24.66.65:38629 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
Aug 12 05:25:52 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:52 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:52.514622 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:52.509299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:56903)
0812 05:25:52.509480 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:52.509494 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:52.509568 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:52.509767 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:52.509768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:52.509950 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:52.510005 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:52.510006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.510719 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.510720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:52.511395 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:52.511398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.511779 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.511780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:52.511833 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:52.512349 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:52.512353 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:52.514558 (+ 2205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":110,"threads_started":1}
W20260812 05:25:52.514736 2149 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:52.509350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:56903 (local address 127.24.66.126:39389)
0812 05:25:52.509440 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:52.509442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:52.509563 (+ 121us) server_negotiation.cc:407] Connection header received
0812 05:25:52.509638 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:52.509639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:52.509651 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:52.509698 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:52.510079 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.510609 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.511497 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.511688 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.511745 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:52.514648 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:56903: BlockingRecv error: recv got EOF from 127.24.66.66:56903 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 05:25:52 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:52 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:52.648731 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:52.643500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:46325)
0812 05:25:52.643638 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:52.643650 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:52.643717 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:52.643866 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:52.643866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:52.644034 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:52.644086 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:52.644087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.644782 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.644783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:52.645329 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:52.645331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.645702 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.645703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:52.645743 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:52.646423 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:52.646429 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:52.648629 (+ 2200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
W20260812 05:25:52.648790 2149 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:52.643577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:46325 (local address 127.24.66.126:39389)
0812 05:25:52.643647 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:52.643649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:52.643706 (+ 57us) server_negotiation.cc:407] Connection header received
0812 05:25:52.643789 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:52.643790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:52.643802 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:52.643848 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:52.644172 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.644688 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.645418 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:52.645623 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:52.645689 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:52.648725 (+ 3036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:46325: BlockingRecv error: recv got EOF from 127.24.66.67:46325 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 05:25:53 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:53 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:53.370908 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:53.365915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:38851)
0812 05:25:53.366128 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:53.366140 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:53.366216 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:53.366439 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:53.366440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:53.366628 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:53.366686 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:53.366688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.367376 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.367377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:53.367939 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:53.367941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.368280 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.368281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:53.368326 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:53.368898 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:53.368903 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:53.370845 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":115,"threads_started":1}
W20260812 05:25:53.371031 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:53.366006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:38851 (local address 127.24.66.126:39389)
0812 05:25:53.366136 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:53.366138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:53.366209 (+ 71us) server_negotiation.cc:407] Connection header received
0812 05:25:53.366322 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:53.366323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:53.366358 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:53.366424 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:53.366764 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.367292 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.368016 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.368210 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.368264 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:53.370953 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:38851: BlockingRecv error: recv got EOF from 127.24.66.65:38851 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
Aug 12 05:25:53 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:53 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:53.520881 2154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:53.515330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:49395)
0812 05:25:53.515471 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:53.515487 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:53.515551 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:53.515715 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:53.515715 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:53.515903 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:53.515957 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:53.515959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.516717 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.516719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:53.517532 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:53.517535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.517908 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.517908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:53.517947 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:53.518607 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:53.518618 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:53.520800 (+ 2182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
W20260812 05:25:53.520979 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:53.515384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:49395 (local address 127.24.66.126:39389)
0812 05:25:53.515472 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:53.515480 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:53.515535 (+ 55us) server_negotiation.cc:407] Connection header received
0812 05:25:53.515605 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:53.515606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:53.515619 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:53.515682 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:53.516036 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.516595 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.517633 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.517837 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.517902 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:53.520928 (+ 3026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:49395: BlockingRecv error: recv got EOF from 127.24.66.66:49395 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 12 05:25:53 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:53 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:53.654814 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:53.649363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:33961)
0812 05:25:53.649568 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:53.649584 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:53.649683 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:53.649893 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:53.649894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:53.650087 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:53.650137 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:53.650139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.650877 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.650878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:53.651479 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:53.651482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.651860 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.651861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:53.651922 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:53.652516 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:53.652521 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:53.654696 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":101,"threads_started":1}
W20260812 05:25:53.654925 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:53.649540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:33961 (local address 127.24.66.126:39389)
0812 05:25:53.649593 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:53.649595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:53.649665 (+ 70us) server_negotiation.cc:407] Connection header received
0812 05:25:53.649760 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:53.649761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:53.649782 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:53.649883 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:53.650221 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.650788 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.651564 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:53.651796 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:53.651861 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:53.654806 (+ 2945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:33961: BlockingRecv error: recv got EOF from 127.24.66.67:33961 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 12 05:25:54 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:54 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:54.380949 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:54.371565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:57401)
0812 05:25:54.371723 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:54.371737 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:54.371822 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:54.371972 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:54.371973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:54.372148 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:54.372199 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:54.372201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.372917 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.372918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:54.373432 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:54.373434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.373827 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.373828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:54.373866 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:54.378691 (+ 4825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:54.378697 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:54.380868 (+ 2171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
W20260812 05:25:54.381141 2157 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:54.371626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:57401 (local address 127.24.66.126:39389)
0812 05:25:54.371780 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:54.371782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:54.371792 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:25:54.371880 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:54.371881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:54.371893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:54.371960 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:54.372264 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.372840 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.373507 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.373753 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.373817 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:54.381026 (+ 7209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:57401: BlockingRecv error: recv got EOF from 127.24.66.65:57401 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
Aug 12 05:25:54 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:54 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:54.526778 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:54.521579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:47213)
0812 05:25:54.521747 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:54.521762 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:54.521832 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:54.521989 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:54.521990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:54.522167 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:54.522217 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:54.522218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.523026 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.523028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:54.523726 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:54.523729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.524160 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.524162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:54.524235 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:54.524709 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:54.524714 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:54.526668 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
W20260812 05:25:54.526902 2157 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:54.521680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:47213 (local address 127.24.66.126:39389)
0812 05:25:54.521754 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:54.521760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:54.521811 (+ 51us) server_negotiation.cc:407] Connection header received
0812 05:25:54.521891 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:54.521891 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:54.521904 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:54.521971 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:54.522286 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.522903 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.523835 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.524069 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.524152 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:54.526836 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:47213: BlockingRecv error: recv got EOF from 127.24.66.66:47213 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 05:25:54 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:54 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:54.661054 2159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:54.655541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:57395)
0812 05:25:54.655713 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:54.655732 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:54.655790 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:54.656017 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:54.656018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:54.656265 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:54.656317 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:54.656320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.657216 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.657216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:54.657771 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:54.657772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.658165 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.658166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:54.658269 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:54.658861 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:54.658867 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:54.660941 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
W20260812 05:25:54.661116 2157 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:54.655687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:57395 (local address 127.24.66.126:39389)
0812 05:25:54.655751 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:54.655759 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:54.655774 (+ 15us) server_negotiation.cc:407] Connection header received
0812 05:25:54.655850 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:54.655851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:54.655865 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:54.655934 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:54.656383 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.657142 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.657841 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:54.658063 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:54.658227 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:54.661056 (+ 2829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:57395: BlockingRecv error: recv got EOF from 127.24.66.67:57395 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 05:25:55 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:55 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:55.386715 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:55.381529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:60233)
0812 05:25:55.381684 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:55.381697 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:55.381752 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:55.381915 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:55.381916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:55.382108 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:55.382167 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:55.382169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.382890 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.382891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:55.383496 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:55.383498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.383890 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.383891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:55.383938 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:55.384469 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:55.384475 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:55.386647 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
W20260812 05:25:55.386884 2161 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:55.381591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:60233 (local address 127.24.66.126:39389)
0812 05:25:55.381729 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:55.381731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:55.381740 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:55.381815 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:55.381816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:55.381827 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:55.381898 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:55.382238 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.382786 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.383577 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.383786 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.383875 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:55.386791 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:60233: BlockingRecv error: recv got EOF from 127.24.66.65:60233 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
Aug 12 05:25:55 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:55 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:55.532591 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:55.527385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:37225)
0812 05:25:55.527534 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:55.527552 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:55.527612 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:55.527763 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:55.527764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:55.527938 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:55.527987 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:55.527989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.528786 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.528788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:55.529433 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:55.529435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.529861 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.529862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:55.529939 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:55.530602 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:55.530607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:55.532531 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
W20260812 05:25:55.532724 2161 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:55.527492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:37225 (local address 127.24.66.126:39389)
0812 05:25:55.527548 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:55.527550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:55.527596 (+ 46us) server_negotiation.cc:407] Connection header received
0812 05:25:55.527686 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:55.527686 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:55.527699 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:55.527750 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:55.528053 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.528670 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.529513 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.529782 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.529847 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:55.532636 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:37225: BlockingRecv error: recv got EOF from 127.24.66.66:37225 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 12 05:25:55 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:55 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:55.666963 2163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:55.661784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:57117)
0812 05:25:55.661974 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:55.661985 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:55.662072 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:55.662241 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:55.662241 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:55.662452 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:55.662502 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:55.662504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.663269 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.663270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:55.663811 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:55.663813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.664199 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.664200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:55.664265 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:55.664720 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:55.664724 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:55.666856 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":104,"threads_started":1}
W20260812 05:25:55.667016 2161 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:55.661856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:57117 (local address 127.24.66.126:39389)
0812 05:25:55.661926 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:55.661928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:55.662049 (+ 121us) server_negotiation.cc:407] Connection header received
0812 05:25:55.662143 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:55.662148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:55.662168 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:55.662232 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:55.662583 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.663090 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.663895 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:55.664116 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:55.664197 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:55.666947 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:57117: BlockingRecv error: recv got EOF from 127.24.66.67:57117 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 05:25:56 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:56 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:56.392985 2164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:56.387454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:39985)
0812 05:25:56.387609 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:56.387623 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:56.387694 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:56.387909 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:56.387909 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:56.388203 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:56.388259 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:56.388261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.389230 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.389232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:56.389935 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:56.389938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.390433 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.390434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:56.390483 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:56.390943 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:56.390948 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:56.392910 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
W20260812 05:25:56.393152 2165 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:56.387541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:39985 (local address 127.24.66.126:39389)
0812 05:25:56.387752 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:56.387755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:56.387767 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:25:56.387796 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:56.387797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:56.387809 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:56.387890 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:56.388372 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.389094 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.390053 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.390362 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.390427 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:56.393068 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:39985: BlockingRecv error: recv got EOF from 127.24.66.65:39985 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":103,"threads_started":1}
Aug 12 05:25:56 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:56 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:56.538463 2166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:56.533220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:38661)
0812 05:25:56.533371 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:56.533486 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:56.533616 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:56.533797 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:56.533798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:56.534042 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:56.534100 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:56.534102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.534894 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.534895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:56.535459 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:56.535461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.535818 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.535819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:56.535864 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:56.536414 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:56.536419 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:56.538382 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":113,"threads_started":1}
W20260812 05:25:56.538564 2165 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:56.533473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:38661 (local address 127.24.66.126:39389)
0812 05:25:56.533545 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:56.533548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:56.533557 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:56.533688 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:56.533690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:56.533704 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:56.533777 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:56.534256 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.534803 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.535535 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.535745 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.535806 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:56.538497 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:38661: BlockingRecv error: recv got EOF from 127.24.66.66:38661 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 05:25:56 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:56 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:56.672807 2167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:56.667485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:54265)
0812 05:25:56.667662 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:56.667672 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:56.667732 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:56.667913 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:56.667914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:56.668092 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:56.668233 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:56.668235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.668902 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.668902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:56.669532 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:56.669534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.669901 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.669903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:56.669956 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:56.670630 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:56.670636 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:56.672735 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":111,"threads_started":1}
W20260812 05:25:56.672915 2165 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:56.667563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:54265 (local address 127.24.66.126:39389)
0812 05:25:56.667607 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:56.667609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:56.667728 (+ 119us) server_negotiation.cc:407] Connection header received
0812 05:25:56.667785 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:56.667786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:56.667798 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:56.667861 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:56.668336 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.668830 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.669613 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:56.669803 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:56.669871 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:56.672839 (+ 2968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:54265: BlockingRecv error: recv got EOF from 127.24.66.67:54265 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 12 05:25:57 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:57 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:57.398893 2169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:57.393520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:49699)
0812 05:25:57.393709 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:57.393723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:57.393799 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:57.393986 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:57.393987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:57.394183 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:57.394236 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:57.394237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.394996 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.394997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:57.395660 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:57.395662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.396068 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.396070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:57.396150 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:57.396608 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:57.396614 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:57.398790 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":81,"threads_started":1}
W20260812 05:25:57.398957 2170 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:57.393639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:49699 (local address 127.24.66.126:39389)
0812 05:25:57.393797 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:57.393799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:57.393836 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:25:57.393860 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:57.393861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:57.393871 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:57.393935 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:57.394311 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.394906 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.395782 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.395993 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.396069 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:57.398885 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:49699: BlockingRecv error: recv got EOF from 127.24.66.65:49699 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
Aug 12 05:25:57 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:57 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:57.544306 2171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:57.539053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:39647)
0812 05:25:57.539243 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:57.539253 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:57.539330 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:57.539496 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:57.539497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:57.539691 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:57.539738 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:57.539740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.540459 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.540460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:57.540993 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:57.540995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.541366 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.541366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:57.541426 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:57.541939 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:57.541951 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:57.544209 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":91,"threads_started":1}
W20260812 05:25:57.544411 2170 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:57.539112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:39647 (local address 127.24.66.126:39389)
0812 05:25:57.539184 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:57.539186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:57.539319 (+ 133us) server_negotiation.cc:407] Connection header received
0812 05:25:57.539403 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:57.539404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:57.539422 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:57.539487 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:57.539816 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.540364 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.541076 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.541274 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.541364 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:57.544344 (+ 2980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:39647: BlockingRecv error: recv got EOF from 127.24.66.66:39647 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 05:25:57 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:57 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.67@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:57.678879 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:57.673405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.67:48043)
0812 05:25:57.673579 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:57.673591 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:57.673667 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:57.673841 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:57.673843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:57.674044 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:57.674098 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:57.674100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.675068 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.675069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:57.675744 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:57.675748 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.676110 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.676110 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:57.676170 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:57.676677 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:57.676682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:57.678775 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
W20260812 05:25:57.679004 2170 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:57.673487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:48043 (local address 127.24.66.126:39389)
0812 05:25:57.673578 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:57.673580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:57.673683 (+ 103us) server_negotiation.cc:407] Connection header received
0812 05:25:57.673717 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:57.673718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:57.673733 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:57.673798 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:57.674383 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.674950 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.675832 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:57.676022 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:57.676108 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:57.678896 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.67:48043: BlockingRecv error: recv got EOF from 127.24.66.67:48043 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 05:25:58 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:58 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.65@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:58.405300 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:58.399511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.65:60415)
0812 05:25:58.399675 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:58.399691 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:58.399744 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:58.399897 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:58.399898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:58.400064 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:58.400161 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:58.400164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.401308 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.401318 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:58.401916 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:58.401918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.402430 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.402432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:58.402485 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:58.402923 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:58.402928 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:58.405235 (+ 2307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":116,"threads_started":1}
W20260812 05:25:58.405390 2175 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:58.399597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:60415 (local address 127.24.66.126:39389)
0812 05:25:58.399758 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:58.399761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:58.399770 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:25:58.399800 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:58.399800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:58.399811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:58.399875 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:58.400235 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.401099 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.402018 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.402296 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.402409 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:58.405303 (+ 2894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.65:60415: BlockingRecv error: recv got EOF from 127.24.66.65:60415 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
Aug 12 05:25:58 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 05:25:58 dist-test-slave-zpfg krb5kdc[1022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.66.66@KRBTEST.COM for oryx/127.24.66.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 05:25:58.550637 2176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:58.544963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39389 (local address 127.24.66.66:44659)
0812 05:25:58.545149 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:58.545163 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:58.545251 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:58.545436 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:58.545437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:58.545604 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:58.545712 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:58.545714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.546428 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.546430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:58.547243 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:58.547246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.547652 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.547653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:58.547714 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:58.548268 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:25:58.548273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:58.550543 (+ 2270us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.66.126:39389: Server oryx/127.24.66.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
W20260812 05:25:58.550796 2175 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:58.545027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:44659 (local address 127.24.66.126:39389)
0812 05:25:58.545093 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:58.545096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:58.545237 (+ 141us) server_negotiation.cc:407] Connection header received
0812 05:25:58.545345 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:58.545346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:58.545359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:58.545414 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:58.545774 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.546269 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.547362 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.547559 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.547617 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:58.550702 (+ 3085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.66.66:44659: BlockingRecv error: recv got EOF from 127.24.66.66:44659 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 05:25:58.585321 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 1650
I20260812 05:25:58.591995 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 1815
I20260812 05:25:58.598400 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 1965
I20260812 05:25:58.604481 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 1575
2026-08-12T05:25:58Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17743 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T05:25:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:58Z Disabled control of system clock
I20260812 05:25:58.636049 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39873
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:44713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:39873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 05:25:58.720296 2181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:58.720523 2181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:58.720544 2181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:58.721664 2181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:58.721701 2181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:58.721714 2181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:58.721726 2181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:58.721745 2181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:58.723044 2181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:39873
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:58.723249 2181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:58.723496 2181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:25:58.729024 2181 server_base.cc:1034] running on GCE node
W20260812 05:25:58.728956 2188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:58.729001 2190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:58.728979 2186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:58.729574 2181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:58.729921 2181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:58.731158 2181 hybrid_clock.cc:648] HybridClock initialized: now 1786512358731149 us; error 36 us; skew 500 ppm
I20260812 05:25:58.732524 2181 webserver.cc:460] Webserver started at http://127.24.66.126:34627/ using document root <none> and password file <none>
I20260812 05:25:58.732782 2181 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:58.732854 2181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:58.733009 2181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:58.734157 2181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "6a3994f1efbf4df4bc9ede4f034705cb"
format_stamp: "Formatted at 2026-08-12 05:25:58 on dist-test-slave-zpfg"
server_key: "829a7b08b24626f98a2bc59101774e88"
server_key_iv: "357535143a06543ef66c2ce9ff5975bd"
server_key_version: "encryptionkey@0"
I20260812 05:25:58.734601 2181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "6a3994f1efbf4df4bc9ede4f034705cb"
format_stamp: "Formatted at 2026-08-12 05:25:58 on dist-test-slave-zpfg"
server_key: "829a7b08b24626f98a2bc59101774e88"
server_key_iv: "357535143a06543ef66c2ce9ff5975bd"
server_key_version: "encryptionkey@0"
I20260812 05:25:58.735965 2181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:58.736832 2196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:58.737027 2181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:58.737133 2181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "6a3994f1efbf4df4bc9ede4f034705cb"
format_stamp: "Formatted at 2026-08-12 05:25:58 on dist-test-slave-zpfg"
server_key: "829a7b08b24626f98a2bc59101774e88"
server_key_iv: "357535143a06543ef66c2ce9ff5975bd"
server_key_version: "encryptionkey@0"
I20260812 05:25:58.737205 2181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:58.767120 2181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:58.767493 2181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:58.767650 2181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:58.772051 2181 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:39873
I20260812 05:25:58.772126 2250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:39873 every 8 connection(s)
I20260812 05:25:58.772495 2181 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:58.773272 2251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:58.777108 2251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb: Bootstrap starting.
I20260812 05:25:58.777904 2251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:58.778270 2251 log.cc:824] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:58.779376 2251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb: No bootstrap required, opened a new log
I20260812 05:25:58.781925 2251 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3994f1efbf4df4bc9ede4f034705cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39873 } }
I20260812 05:25:58.782130 2251 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:58.782171 2251 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3994f1efbf4df4bc9ede4f034705cb, State: Initialized, Role: FOLLOWER
I20260812 05:25:58.782363 2251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3994f1efbf4df4bc9ede4f034705cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39873 } }
I20260812 05:25:58.782460 2251 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:58.782500 2251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:58.782546 2251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:58.782557 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 2181
I20260812 05:25:58.782655 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:58.782876 24841 external_mini_cluster.cc:1428] Setting key a8b05122986c0cd3a001efbb2b5d64a2
I20260812 05:25:58.783383 2251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3994f1efbf4df4bc9ede4f034705cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39873 } }
I20260812 05:25:58.783519 2251 leader_election.cc:302] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a3994f1efbf4df4bc9ede4f034705cb; no voters:
I20260812 05:25:58.783706 2251 leader_election.cc:288] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:58.783977 2251 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:58.784155 2257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:58.784300 2257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [term 1 LEADER]: Becoming Leader. State: Replica: 6a3994f1efbf4df4bc9ede4f034705cb, State: Running, Role: LEADER
I20260812 05:25:58.784387 2257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3994f1efbf4df4bc9ede4f034705cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39873 } }
I20260812 05:25:58.784754 2257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a3994f1efbf4df4bc9ede4f034705cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3994f1efbf4df4bc9ede4f034705cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39873 } } }
I20260812 05:25:58.784956 2257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:58.785204 2257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a3994f1efbf4df4bc9ede4f034705cb. Latest consensus state: current_term: 1 leader_uuid: "6a3994f1efbf4df4bc9ede4f034705cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3994f1efbf4df4bc9ede4f034705cb" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39873 } } }
I20260812 05:25:58.785329 2257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:58.785535 2268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:58.786193 2268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:58.788740 2268 catalog_manager.cc:1347] Generated new cluster ID: b5a149fbefb340fa96139937a1e9e183
I20260812 05:25:58.788864 2268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:58.790110 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:58.783759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.24.66.126:39873)
0812 05:25:58.784156 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:58.784166 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:58.784203 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:25:58.784512 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:58.784520 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:58.784597 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:58.784744 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:58.785076 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.786648 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.787747 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.788375 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.788844 (+ 469us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:58.789289 (+ 445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:58.789293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:58.789294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:58.789546 (+ 252us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:58.789592 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:58.789618 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:58.789683 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:58.789761 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 05:25:58.789853 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":72,"threads_started":1}
I20260812 05:25:58.814009 2268 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:58.814667 2268 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:58.826730 2268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a3994f1efbf4df4bc9ede4f034705cb: Generated new TSK 0
I20260812 05:25:58.827008 2268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:25:58.838697 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39873
--builtin_ntp_servers=127.24.66.84:44713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 05:25:58.929437 2278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:58.929787 2278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:58.929834 2278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:58.931421 2278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:58.931483 2278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:58.931514 2278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:58.933009 2278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:58.933324 2278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:58.933676 2278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:58.939451 2286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:58.939481 2284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:58.939481 2283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:58.940344 2278 server_base.cc:1034] running on GCE node
I20260812 05:25:58.940587 2278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:58.940909 2278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:58.942087 2278 hybrid_clock.cc:648] HybridClock initialized: now 1786512358942075 us; error 32 us; skew 500 ppm
I20260812 05:25:58.943671 2278 webserver.cc:460] Webserver started at http://127.24.66.65:45715/ using document root <none> and password file <none>
I20260812 05:25:58.943961 2278 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:58.944036 2278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:58.944202 2278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:58.945401 2278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "f9d38a453d6341319424c8205b3eb762"
format_stamp: "Formatted at 2026-08-12 05:25:58 on dist-test-slave-zpfg"
server_key: "aa4a11e114f98e1f52503f9e9e95924d"
server_key_iv: "de5f2fb7adba14c5009232b32dce34a0"
server_key_version: "encryptionkey@0"
I20260812 05:25:58.945776 2278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "f9d38a453d6341319424c8205b3eb762"
format_stamp: "Formatted at 2026-08-12 05:25:58 on dist-test-slave-zpfg"
server_key: "aa4a11e114f98e1f52503f9e9e95924d"
server_key_iv: "de5f2fb7adba14c5009232b32dce34a0"
server_key_version: "encryptionkey@0"
I20260812 05:25:58.947333 2278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:58.948103 2292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:58.948346 2278 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:58.948450 2278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "f9d38a453d6341319424c8205b3eb762"
format_stamp: "Formatted at 2026-08-12 05:25:58 on dist-test-slave-zpfg"
server_key: "aa4a11e114f98e1f52503f9e9e95924d"
server_key_iv: "de5f2fb7adba14c5009232b32dce34a0"
server_key_version: "encryptionkey@0"
I20260812 05:25:58.948540 2278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:58.958176 2278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:58.958649 2278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:58.958828 2278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:58.959091 2278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:58.959471 2278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:58.959506 2278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:58.959532 2278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:58.959545 2278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:58.966722 2278 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:34491
I20260812 05:25:58.966861 2417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:34491 every 8 connection(s)
I20260812 05:25:58.967226 2278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:58.972978 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:58.968056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:54287 (local address 127.24.66.126:39873)
0812 05:25:58.968149 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:58.968151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:58.968395 (+ 244us) server_negotiation.cc:407] Connection header received
0812 05:25:58.968910 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:58.968911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:58.968924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:58.968982 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:58.969682 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.970191 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.971560 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.971761 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.971827 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:58.972705 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:58.972707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:58.972707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:58.972714 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:58.972728 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:58.972733 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:58.972777 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:58.972876 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 05:25:58.972911 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 05:25:58.973184 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:58.968006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39873 (local address 127.24.66.65:54287)
0812 05:25:58.968235 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:58.968285 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:58.968813 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:58.969043 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:58.969046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:58.969055 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:58.969585 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:58.969589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.970465 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.970467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:58.971461 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:58.971464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:58.971847 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:58.971848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:58.971964 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:58.972527 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:58.972527 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:58.972560 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:58.972585 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:58.972793 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:58.972799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:58.972802 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:58.972859 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:25:58.972948 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260812 05:25:58.973758 2418 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39873
I20260812 05:25:58.973883 2418 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:58.974089 2418 heartbeater.cc:511] Master 127.24.66.126:39873 requested a full tablet report, sending...
I20260812 05:25:58.974709 2213 ts_manager.cc:194] Registered new tserver with Master: f9d38a453d6341319424c8205b3eb762 (127.24.66.65:34491)
I20260812 05:25:58.975291 2213 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.65:54287
I20260812 05:25:58.975409 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 2278
I20260812 05:25:58.975535 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:58.975770 24841 external_mini_cluster.cc:1428] Setting key 80603bcb3ed3a435787a15b4b4bfb867
I20260812 05:25:58.976804 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39873
--builtin_ntp_servers=127.24.66.84:44713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 05:25:59.068153 2421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:59.068394 2421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:59.068418 2421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:59.069675 2421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:59.069723 2421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:59.069751 2421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:59.071741 2421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:59.072072 2421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:59.072467 2421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:59.078696 2427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:59.078681 2429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:59.078709 2426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:59.079033 2421 server_base.cc:1034] running on GCE node
I20260812 05:25:59.079255 2421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:59.079553 2421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:59.080745 2421 hybrid_clock.cc:648] HybridClock initialized: now 1786512359080716 us; error 55 us; skew 500 ppm
I20260812 05:25:59.082209 2421 webserver.cc:460] Webserver started at http://127.24.66.66:35587/ using document root <none> and password file <none>
I20260812 05:25:59.082509 2421 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:59.082587 2421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:59.082747 2421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:59.083936 2421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "d6b5b22e4c3941bd9f9ce2f4e9de9914"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "6c6d6bc8cad970958cc6eeca3e4877fa"
server_key_iv: "d9594cd80e377b26a2702c8828857bf2"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.084338 2421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "d6b5b22e4c3941bd9f9ce2f4e9de9914"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "6c6d6bc8cad970958cc6eeca3e4877fa"
server_key_iv: "d9594cd80e377b26a2702c8828857bf2"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.086242 2421 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 05:25:59.087304 2439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.087519 2421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:59.087632 2421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "d6b5b22e4c3941bd9f9ce2f4e9de9914"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "6c6d6bc8cad970958cc6eeca3e4877fa"
server_key_iv: "d9594cd80e377b26a2702c8828857bf2"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.087729 2421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:59.109964 2421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:59.110387 2421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:59.110566 2421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:59.110833 2421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:59.111195 2421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:59.111251 2421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.111286 2421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:59.111315 2421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.117780 2421 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:38633
I20260812 05:25:59.117827 2560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:38633 every 8 connection(s)
I20260812 05:25:59.118218 2421 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:59.123709 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:59.119035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:54749 (local address 127.24.66.126:39873)
0812 05:25:59.119214 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:59.119217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:59.119396 (+ 179us) server_negotiation.cc:407] Connection header received
0812 05:25:59.119955 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:59.119957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:59.119970 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:59.120024 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:59.120709 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:59.121227 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:59.122240 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:59.122459 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:59.122547 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:59.123397 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:59.123399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:59.123399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:59.123405 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:59.123422 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:59.123426 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:59.123498 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:59.123592 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 05:25:59.123623 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:25:59.123895 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 2421
I20260812 05:25:59.123993 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:59.124025 2563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:59.119027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39873 (local address 127.24.66.66:54749)
0812 05:25:59.119238 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:59.119283 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:59.119840 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:59.120102 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:59.120107 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:59.120124 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:59.120601 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:59.120605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:59.121330 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:59.121331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:59.122130 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:59.122133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:59.122558 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:59.122560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:59.122700 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:59.123252 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:59.123253 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:59.123283 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:59.123306 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:59.123498 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:59.123505 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:59.123508 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:59.123664 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 05:25:59.123790 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260812 05:25:59.124214 24841 external_mini_cluster.cc:1428] Setting key 464741e2e0f35abfa6ecc4e014625dd0
I20260812 05:25:59.124547 2561 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39873
I20260812 05:25:59.124673 2561 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:59.124898 2561 heartbeater.cc:511] Master 127.24.66.126:39873 requested a full tablet report, sending...
I20260812 05:25:59.125319 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39873
--builtin_ntp_servers=127.24.66.84:44713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 05:25:59.125344 2213 ts_manager.cc:194] Registered new tserver with Master: d6b5b22e4c3941bd9f9ce2f4e9de9914 (127.24.66.66:38633)
I20260812 05:25:59.125844 2213 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:54749
W20260812 05:25:59.218746 2564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:59.219023 2564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:59.219050 2564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:59.220399 2564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:59.220446 2564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:59.220477 2564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:59.222060 2564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:59.222318 2564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:59.222647 2564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:59.228412 2572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:59.228412 2569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:59.228470 2570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:59.228950 2564 server_base.cc:1034] running on GCE node
I20260812 05:25:59.229224 2564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:59.229478 2564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:59.230629 2564 hybrid_clock.cc:648] HybridClock initialized: now 1786512359230617 us; error 30 us; skew 500 ppm
I20260812 05:25:59.232127 2564 webserver.cc:460] Webserver started at http://127.24.66.67:43855/ using document root <none> and password file <none>
I20260812 05:25:59.232436 2564 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:59.232492 2564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:59.232668 2564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:59.233908 2564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "469a11c854ab4a37b07aab29d6ef541e"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "9583b89dc437caa46efa23db7c5d847d"
server_key_iv: "f07ff1240165461fa0e28c881f75b9f5"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.234431 2564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "469a11c854ab4a37b07aab29d6ef541e"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "9583b89dc437caa46efa23db7c5d847d"
server_key_iv: "f07ff1240165461fa0e28c881f75b9f5"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.235996 2564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:59.236905 2578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.237207 2564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:59.237330 2564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "469a11c854ab4a37b07aab29d6ef541e"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "9583b89dc437caa46efa23db7c5d847d"
server_key_iv: "f07ff1240165461fa0e28c881f75b9f5"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.237413 2564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:59.276572 2564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:59.277014 2564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:59.277222 2564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:59.277547 2564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:59.277967 2564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:59.278033 2564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.278085 2564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:59.278112 2564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.285646 2564 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:40879
I20260812 05:25:59.285710 2702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:40879 every 8 connection(s)
I20260812 05:25:59.286288 2564 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:59.292702 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 2564
I20260812 05:25:59.292814 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:59.292810 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:59.287260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:59489 (local address 127.24.66.126:39873)
0812 05:25:59.287368 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:59.287370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:59.287563 (+ 193us) server_negotiation.cc:407] Connection header received
0812 05:25:59.288078 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:59.288079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:59.288092 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:59.288185 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:59.289042 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:59.289723 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:59.291079 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:25:59.291311 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:25:59.291381 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:59.292379 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:59.292381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:59.292381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:59.292387 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:59.292404 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:59.292408 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:59.292460 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:59.292712 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 05:25:59.292745 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 05:25:59.293031 24841 external_mini_cluster.cc:1428] Setting key bfa992b7ee1de08e44d009f15677ae57
I20260812 05:25:59.293116 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:59.287178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39873 (local address 127.24.66.67:59489)
0812 05:25:59.287422 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:59.287465 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:59.287977 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:59.288301 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:59.288305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:59.288317 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:59.288936 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:59.288942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:59.289886 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:59.289887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:59.290996 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:25:59.290999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:25:59.291398 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:25:59.291399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:25:59.291515 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:25:59.292177 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:59.292177 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:59.292212 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:59.292253 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:59.292524 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:59.292533 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:59.292537 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:25:59.292717 (+ 180us) client_negotiation.cc:240] Negotiation successful
0812 05:25:59.292857 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
I20260812 05:25:59.293723 2703 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39873
I20260812 05:25:59.293865 2703 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:59.294061 2703 heartbeater.cc:511] Master 127.24.66.126:39873 requested a full tablet report, sending...
I20260812 05:25:59.294493 2212 ts_manager.cc:194] Registered new tserver with Master: 469a11c854ab4a37b07aab29d6ef541e (127.24.66.67:40879)
I20260812 05:25:59.294899 2212 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:59489
I20260812 05:25:59.305451 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 05:25:59.307648 2258 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:59.307086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53510 (local address 127.24.66.126:39873)
0812 05:25:59.307160 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:59.307162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:59.307264 (+ 102us) server_negotiation.cc:407] Connection header received
0812 05:25:59.307330 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:59.307331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:59.307342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:59.307389 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:59.307556 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53510: BlockingRecv error: recv got EOF from 127.0.0.1:53510 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:25:59.307726 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 2278
I20260812 05:25:59.317369 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 2421
I20260812 05:25:59.324208 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 2564
I20260812 05:25:59.330454 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 2181
2026-08-12T05:25:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (724 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T05:25:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:25:59Z Disabled control of system clock
I20260812 05:25:59.364434 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40767
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:41135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:40767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 05:25:59.447422 2714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:59.447688 2714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:59.447731 2714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:59.448942 2714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:25:59.449023 2714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:59.449043 2714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:59.449064 2714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:25:59.449096 2714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:25:59.450582 2714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:40767
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:59.450917 2714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:59.451341 2714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:59.456847 2719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:59.456880 2723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:59.456858 2721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:59.456959 2714 server_base.cc:1034] running on GCE node
I20260812 05:25:59.457376 2714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:59.457636 2714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:59.458782 2714 hybrid_clock.cc:648] HybridClock initialized: now 1786512359458767 us; error 30 us; skew 500 ppm
I20260812 05:25:59.465602 2714 webserver.cc:460] Webserver started at http://127.24.66.126:40275/ using document root <none> and password file <none>
I20260812 05:25:59.465915 2714 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:59.465973 2714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:59.466152 2714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:59.467439 2714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "da310cdde7d649c294281e58ace3b51c"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "18a1107b7590696ca80bc2226f194a5d"
server_key_iv: "40ff670dcbec9d99a28d8f1b27f44694"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.467813 2714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "da310cdde7d649c294281e58ace3b51c"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "18a1107b7590696ca80bc2226f194a5d"
server_key_iv: "40ff670dcbec9d99a28d8f1b27f44694"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.469204 2714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:25:59.470043 2730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.470391 2714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:59.470510 2714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "da310cdde7d649c294281e58ace3b51c"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "18a1107b7590696ca80bc2226f194a5d"
server_key_iv: "40ff670dcbec9d99a28d8f1b27f44694"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.470597 2714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:59.474210 2714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:59.474565 2714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:59.474710 2714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:59.479030 2714 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:40767
I20260812 05:25:59.479132 2786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:40767 every 8 connection(s)
I20260812 05:25:59.479480 2714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:25:59.480201 2787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:25:59.483362 2787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c: Bootstrap starting.
I20260812 05:25:59.484014 2787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c: Neither blocks nor log segments found. Creating new log.
I20260812 05:25:59.484361 2787 log.cc:824] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:25:59.485157 2787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c: No bootstrap required, opened a new log
I20260812 05:25:59.487011 2787 raft_consensus.cc:348] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da310cdde7d649c294281e58ace3b51c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40767 } }
I20260812 05:25:59.487167 2787 raft_consensus.cc:374] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:25:59.487190 2787 raft_consensus.cc:729] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da310cdde7d649c294281e58ace3b51c, State: Initialized, Role: FOLLOWER
I20260812 05:25:59.487391 2787 consensus_queue.cc:260] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da310cdde7d649c294281e58ace3b51c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40767 } }
I20260812 05:25:59.487465 2787 raft_consensus.cc:388] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:25:59.487521 2787 raft_consensus.cc:482] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:25:59.487573 2787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:25:59.488215 2787 raft_consensus.cc:504] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da310cdde7d649c294281e58ace3b51c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40767 } }
I20260812 05:25:59.488314 2787 leader_election.cc:302] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da310cdde7d649c294281e58ace3b51c; no voters:
I20260812 05:25:59.488536 2787 leader_election.cc:288] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:25:59.488580 2790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:25:59.488831 2790 raft_consensus.cc:686] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [term 1 LEADER]: Becoming Leader. State: Replica: da310cdde7d649c294281e58ace3b51c, State: Running, Role: LEADER
I20260812 05:25:59.488902 2787 sys_catalog.cc:564] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:25:59.488936 2790 consensus_queue.cc:237] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da310cdde7d649c294281e58ace3b51c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40767 } }
I20260812 05:25:59.489742 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 2714
I20260812 05:25:59.489862 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:25:59.490095 24841 external_mini_cluster.cc:1428] Setting key 328b3a515fba43468221e80845336077
I20260812 05:25:59.490203 2792 sys_catalog.cc:455] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [sys.catalog]: SysCatalogTable state changed. Reason: New leader da310cdde7d649c294281e58ace3b51c. Latest consensus state: current_term: 1 leader_uuid: "da310cdde7d649c294281e58ace3b51c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da310cdde7d649c294281e58ace3b51c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40767 } } }
I20260812 05:25:59.490285 2792 sys_catalog.cc:458] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [sys.catalog]: This master's current role is: LEADER
I20260812 05:25:59.490533 2791 sys_catalog.cc:455] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da310cdde7d649c294281e58ace3b51c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da310cdde7d649c294281e58ace3b51c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40767 } } }
I20260812 05:25:59.490638 2791 sys_catalog.cc:458] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c [sys.catalog]: This master's current role is: LEADER
W20260812 05:25:59.491180 2805 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:25:59.491243 2805 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:25:59.491463 2802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:25:59.491676 2802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:25:59.493939 2802 catalog_manager.cc:1347] Generated new cluster ID: dc2d6af2a1d44c96a8252413822d5562
I20260812 05:25:59.494000 2802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:25:59.494920 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:59.492463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38030 (local address 127.24.66.126:40767)
0812 05:25:59.492685 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:59.492696 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:59.492732 (+ 36us) server_negotiation.cc:407] Connection header received
0812 05:25:59.492862 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:59.492869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:59.492915 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:59.493077 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:59.494304 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:59.494308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:59.494308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:59.494313 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:59.494421 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:59.494437 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:59.494530 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:59.494642 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 05:25:59.494767 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260812 05:25:59.506425 2802 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:25:59.506623 2802 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:25:59.513705 2802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da310cdde7d649c294281e58ace3b51c: Generated new TSK 0
I20260812 05:25:59.513952 2802 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:25:59.518193 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40767
--builtin_ntp_servers=127.24.66.84:41135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 05:25:59.613445 2812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:59.613682 2812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:59.613710 2812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:59.615159 2812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:59.615217 2812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:59.615250 2812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:25:59.616863 2812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:59.617182 2812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:59.617542 2812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:59.623353 2817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:59.623387 2818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:59.623426 2812 server_base.cc:1034] running on GCE node
W20260812 05:25:59.623555 2820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:59.623859 2812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:59.624171 2812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:59.625385 2812 hybrid_clock.cc:648] HybridClock initialized: now 1786512359625366 us; error 34 us; skew 500 ppm
I20260812 05:25:59.626889 2812 webserver.cc:460] Webserver started at http://127.24.66.65:46413/ using document root <none> and password file <none>
I20260812 05:25:59.627183 2812 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:59.627256 2812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:59.627422 2812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:59.628607 2812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "e5954ad897894692b3c990c0f384a39f"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "7f31dd6f3fe22ab3bbbe4d7df46c25d0"
server_key_iv: "84060a6244336d97376c5f3025ebd820"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.628989 2812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "e5954ad897894692b3c990c0f384a39f"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "7f31dd6f3fe22ab3bbbe4d7df46c25d0"
server_key_iv: "84060a6244336d97376c5f3025ebd820"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.630540 2812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 05:25:59.631433 2827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.631619 2812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:25:59.631724 2812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "e5954ad897894692b3c990c0f384a39f"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "7f31dd6f3fe22ab3bbbe4d7df46c25d0"
server_key_iv: "84060a6244336d97376c5f3025ebd820"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.631820 2812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:59.635499 2812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:59.635811 2812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:59.635962 2812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:59.636217 2812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:59.636561 2812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:59.636617 2812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.636660 2812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:59.636687 2812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.642846 2812 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:44887
I20260812 05:25:59.642927 2947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:44887 every 8 connection(s)
I20260812 05:25:59.643286 2812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:25:59.644685 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 2812
I20260812 05:25:59.644781 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:25:59.645053 24841 external_mini_cluster.cc:1428] Setting key 551bf74515c8009991946757de460ffa
I20260812 05:25:59.646224 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40767
--builtin_ntp_servers=127.24.66.84:41135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 05:25:59.647109 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:59.644164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:51313 (local address 127.24.66.126:40767)
0812 05:25:59.644235 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:59.644242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:59.644566 (+ 324us) server_negotiation.cc:407] Connection header received
0812 05:25:59.645226 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:59.645227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:59.645236 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:59.645295 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:59.646776 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:59.646777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:59.646777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:59.646783 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:59.646803 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:59.646809 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:59.646890 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:59.646984 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 05:25:59.647003 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:25:59.647230 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:59.644164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40767 (local address 127.24.66.65:51313)
0812 05:25:59.644422 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:59.644459 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:59.645096 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:59.645385 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:59.645388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:59.645397 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:59.646619 (+ 1222us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:59.646619 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:59.646650 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:59.646680 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:59.646894 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:59.646899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:59.646902 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:25:59.646969 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:25:59.647023 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":98,"threads_started":1}
I20260812 05:25:59.647866 2948 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40767
I20260812 05:25:59.648005 2948 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:59.648187 2948 heartbeater.cc:511] Master 127.24.66.126:40767 requested a full tablet report, sending...
I20260812 05:25:59.648737 2748 ts_manager.cc:194] Registered new tserver with Master: e5954ad897894692b3c990c0f384a39f (127.24.66.65:44887)
W20260812 05:25:59.731392 2952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:59.731613 2952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:59.731638 2952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:59.732793 2952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:59.732836 2952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:59.732861 2952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:25:59.734180 2952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:59.734545 2952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:59.734833 2952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:59.740373 2958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:59.740661 2960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:59.740705 2957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:59.740751 2952 server_base.cc:1034] running on GCE node
I20260812 05:25:59.740973 2952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:59.741210 2952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:59.742455 2952 hybrid_clock.cc:648] HybridClock initialized: now 1786512359742427 us; error 42 us; skew 500 ppm
I20260812 05:25:59.743832 2952 webserver.cc:460] Webserver started at http://127.24.66.66:43523/ using document root <none> and password file <none>
I20260812 05:25:59.744073 2952 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:59.744117 2952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:59.744328 2952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:59.745471 2952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "276aa6cac3b74e3287ad79cc67e8a86e"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "10f249b3ca531b7a182940b9da4726b1"
server_key_iv: "403db57a599f4ddc29752ff467372ac8"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.745831 2952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "276aa6cac3b74e3287ad79cc67e8a86e"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "10f249b3ca531b7a182940b9da4726b1"
server_key_iv: "403db57a599f4ddc29752ff467372ac8"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.747411 2952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:59.748338 2966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.748554 2952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:59.748647 2952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "276aa6cac3b74e3287ad79cc67e8a86e"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "10f249b3ca531b7a182940b9da4726b1"
server_key_iv: "403db57a599f4ddc29752ff467372ac8"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.748745 2952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:59.752846 2952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:59.753103 2952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:59.753224 2952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:59.753407 2952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:59.753702 2952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:59.753737 2952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.753759 2952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:59.753773 2952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.760646 2952 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:37869
I20260812 05:25:59.760679 3087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:37869 every 8 connection(s)
I20260812 05:25:59.761044 2952 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:25:59.762388 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 2952
I20260812 05:25:59.762507 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:25:59.762742 24841 external_mini_cluster.cc:1428] Setting key 3ad86399e079315032036a93f06d0c9b
I20260812 05:25:59.763978 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40767
--builtin_ntp_servers=127.24.66.84:41135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 05:25:59.765174 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:59.761815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:41959 (local address 127.24.66.126:40767)
0812 05:25:59.761888 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:59.761891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:59.762152 (+ 261us) server_negotiation.cc:407] Connection header received
0812 05:25:59.762957 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:59.762960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:59.762968 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:59.763050 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:59.764872 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:59.764874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:59.764875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:59.764880 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:59.764900 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:59.764907 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:59.764976 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:59.765059 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 05:25:59.765077 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 05:25:59.765323 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:59.761803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40767 (local address 127.24.66.66:41959)
0812 05:25:59.761995 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:59.762035 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:59.762781 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:59.763332 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:59.763335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:59.763344 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:59.764701 (+ 1357us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:59.764702 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:59.764732 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:59.764766 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:59.764981 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:59.764986 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:59.764988 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:25:59.765052 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 05:25:59.765108 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260812 05:25:59.765944 3089 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40767
I20260812 05:25:59.766055 3089 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:59.766235 3089 heartbeater.cc:511] Master 127.24.66.126:40767 requested a full tablet report, sending...
I20260812 05:25:59.766690 2748 ts_manager.cc:194] Registered new tserver with Master: 276aa6cac3b74e3287ad79cc67e8a86e (127.24.66.66:37869)
W20260812 05:25:59.849608 3092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:25:59.849903 3092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:25:59.849948 3092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:25:59.851315 3092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:25:59.851400 3092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:25:59.851449 3092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:25:59.853211 3092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:25:59.853534 3092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:25:59.853874 3092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:25:59.859532 3097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:59.859560 3101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:25:59.859532 3099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:25:59.859928 3092 server_base.cc:1034] running on GCE node
I20260812 05:25:59.860240 3092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:25:59.860528 3092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:25:59.861721 3092 hybrid_clock.cc:648] HybridClock initialized: now 1786512359861710 us; error 58 us; skew 500 ppm
I20260812 05:25:59.863257 3092 webserver.cc:460] Webserver started at http://127.24.66.67:41275/ using document root <none> and password file <none>
I20260812 05:25:59.863525 3092 fs_manager.cc:359] Metadata directory not provided
I20260812 05:25:59.863577 3092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:25:59.863719 3092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:25:59.864866 3092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "ce70a96af7704ce894e66acb6a520964"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "31538aa0fd39cf2bad751fa7facd8ffe"
server_key_iv: "46d8d7dade59ffcf77e8755f6984f521"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.865239 3092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "ce70a96af7704ce894e66acb6a520964"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "31538aa0fd39cf2bad751fa7facd8ffe"
server_key_iv: "46d8d7dade59ffcf77e8755f6984f521"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.866716 3092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:25:59.867574 3107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.867827 3092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:25:59.867928 3092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "ce70a96af7704ce894e66acb6a520964"
format_stamp: "Formatted at 2026-08-12 05:25:59 on dist-test-slave-zpfg"
server_key: "31538aa0fd39cf2bad751fa7facd8ffe"
server_key_iv: "46d8d7dade59ffcf77e8755f6984f521"
server_key_version: "encryptionkey@0"
I20260812 05:25:59.867990 3092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:25:59.871773 3092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:25:59.872025 3092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:25:59.872123 3092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:25:59.872294 3092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:25:59.872547 3092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:25:59.872577 3092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.872597 3092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:25:59.872610 3092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:25:59.879577 3092 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:34921
I20260812 05:25:59.879616 3227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:34921 every 8 connection(s)
I20260812 05:25:59.880015 3092 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:25:59.880610 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 3092
I20260812 05:25:59.880748 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:25:59.881006 24841 external_mini_cluster.cc:1428] Setting key 1b79a08ad713e501875f358dd0e7a5d4
I20260812 05:25:59.883428 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:59.880837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:60265 (local address 127.24.66.126:40767)
0812 05:25:59.880911 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:59.880913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:59.881207 (+ 294us) server_negotiation.cc:407] Connection header received
0812 05:25:59.881886 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:59.881887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:59.881894 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:59.881972 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:59.883190 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:25:59.883191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:25:59.883192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:25:59.883196 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:25:59.883215 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:25:59.883220 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:25:59.883278 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 05:25:59.883359 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 05:25:59.883372 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:25:59.883641 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:25:59.880855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40767 (local address 127.24.66.67:60265)
0812 05:25:59.881062 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 05:25:59.881106 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:25:59.881737 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:25:59.882052 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:25:59.882057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:25:59.882068 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:25:59.883039 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:25:59.883039 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:25:59.883070 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:25:59.883100 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:25:59.883277 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:25:59.883279 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:25:59.883281 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:25:59.883369 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 05:25:59.883421 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260812 05:25:59.884104 3228 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40767
I20260812 05:25:59.884225 3228 heartbeater.cc:464] Registering TS with master...
I20260812 05:25:59.884383 3228 heartbeater.cc:511] Master 127.24.66.126:40767 requested a full tablet report, sending...
I20260812 05:25:59.884737 2748 ts_manager.cc:194] Registered new tserver with Master: ce70a96af7704ce894e66acb6a520964 (127.24.66.67:34921)
I20260812 05:25:59.893568 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 05:25:59.896003 2810 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:25:59.895251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.24.66.126:40767)
0812 05:25:59.895305 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:25:59.895307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:25:59.895545 (+ 238us) server_negotiation.cc:407] Connection header received
0812 05:25:59.895579 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:25:59.895580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:25:59.895587 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:25:59.895638 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:25:59.895891 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38046: BlockingRecv error: recv got EOF from 127.0.0.1:38046 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 05:25:59.896010 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 2812
I20260812 05:25:59.902462 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 2952
I20260812 05:25:59.908411 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 3092
I20260812 05:25:59.915249 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 2714
2026-08-12T05:25:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (584 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:25:59 dist-test-slave-zpfg krb5kdc[3238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:25:59 dist-test-slave-zpfg krb5kdc[3238](info): set up 2 sockets
Aug 12 05:25:59 dist-test-slave-zpfg krb5kdc[3238](info): commencing operation
krb5kdc: starting...
W20260812 05:26:01.046017 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:26:01 dist-test-slave-zpfg krb5kdc[3238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:26:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:26:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:26:01.189769 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40519
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:40519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:26:01.276628 3252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:26:01.276867 3252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:26:01.276893 3252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:26:01.278187 3252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:26:01.278255 3252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:26:01.278270 3252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:26:01.278285 3252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:26:01.278296 3252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:26:01.279951 3252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:40519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:26:01.280256 3252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:26:01.280512 3252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:26:01.286317 3258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:01.286396 3259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:01.286566 3261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:26:01.286469 3252 server_base.cc:1034] running on GCE node
I20260812 05:26:01.286847 3252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:26:01.287132 3252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:26:01.288285 3252 hybrid_clock.cc:648] HybridClock initialized: now 1786512361288255 us; error 59 us; skew 500 ppm
Aug 12 05:26:01 dist-test-slave-zpfg krb5kdc[3238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:26:01.290969 3252 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:26:01.291451 3252 webserver.cc:460] Webserver started at http://127.24.66.126:36237/ using document root <none> and password file <none>
I20260812 05:26:01.291687 3252 fs_manager.cc:359] Metadata directory not provided
I20260812 05:26:01.291733 3252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:26:01.291839 3252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:26:01.292917 3252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "212cdedf9461450ea65a2ec9a9d42d89"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "7f9cde1239ca7096b001378de27e271a"
server_key_iv: "49dd27dfe084853f20fd00e7781720b2"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.293357 3252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "212cdedf9461450ea65a2ec9a9d42d89"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "7f9cde1239ca7096b001378de27e271a"
server_key_iv: "49dd27dfe084853f20fd00e7781720b2"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.294919 3252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:26:01.295732 3268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:01.295940 3252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:26:01.296038 3252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "212cdedf9461450ea65a2ec9a9d42d89"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "7f9cde1239ca7096b001378de27e271a"
server_key_iv: "49dd27dfe084853f20fd00e7781720b2"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.296109 3252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:26:01.304967 3252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:26:01.307662 3252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:26:01.307809 3252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:26:01.312783 3252 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:40519
I20260812 05:26:01.312913 3324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:40519 every 8 connection(s)
I20260812 05:26:01.313292 3252 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:26:01.314301 3325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:26:01.316100 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 3252
I20260812 05:26:01.316180 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:26:01.316411 24841 external_mini_cluster.cc:1428] Setting key 55b6f43813e05abc9a2b1da7c8540d30
I20260812 05:26:01.317117 3325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89: Bootstrap starting.
I20260812 05:26:01.317914 3325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89: Neither blocks nor log segments found. Creating new log.
I20260812 05:26:01.318250 3325 log.cc:824] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89: Log is configured to *not* fsync() on all Append() calls
I20260812 05:26:01.319093 3325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89: No bootstrap required, opened a new log
I20260812 05:26:01.321414 3325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212cdedf9461450ea65a2ec9a9d42d89" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40519 } }
I20260812 05:26:01.321596 3325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:26:01.321630 3325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 212cdedf9461450ea65a2ec9a9d42d89, State: Initialized, Role: FOLLOWER
I20260812 05:26:01.321758 3325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212cdedf9461450ea65a2ec9a9d42d89" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40519 } }
I20260812 05:26:01.321856 3325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:26:01.321888 3325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:26:01.321943 3325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:26:01.322775 3325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212cdedf9461450ea65a2ec9a9d42d89" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40519 } }
I20260812 05:26:01.322916 3325 leader_election.cc:302] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 212cdedf9461450ea65a2ec9a9d42d89; no voters:
I20260812 05:26:01.323144 3325 leader_election.cc:288] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:26:01.323172 3330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:26:01.323288 3330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [term 1 LEADER]: Becoming Leader. State: Replica: 212cdedf9461450ea65a2ec9a9d42d89, State: Running, Role: LEADER
I20260812 05:26:01.323401 3330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212cdedf9461450ea65a2ec9a9d42d89" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40519 } }
I20260812 05:26:01.323432 3325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:26:01.324092 3331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "212cdedf9461450ea65a2ec9a9d42d89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212cdedf9461450ea65a2ec9a9d42d89" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40519 } } }
I20260812 05:26:01.324295 3331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [sys.catalog]: This master's current role is: LEADER
Aug 12 05:26:01 dist-test-slave-zpfg krb5kdc[3238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:26:01.324080 3330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 212cdedf9461450ea65a2ec9a9d42d89. Latest consensus state: current_term: 1 leader_uuid: "212cdedf9461450ea65a2ec9a9d42d89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "212cdedf9461450ea65a2ec9a9d42d89" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40519 } } }
I20260812 05:26:01.324939 3339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:26:01.325048 3330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89 [sys.catalog]: This master's current role is: LEADER
I20260812 05:26:01.325520 3339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:26:01.327401 3339 catalog_manager.cc:1347] Generated new cluster ID: e3a37c5fd4424b379966baf18c576bd7
I20260812 05:26:01.327457 3339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:26:01.329865 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.317065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37230 (local address 127.24.66.126:40519)
0812 05:26:01.317292 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:01.317302 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:01.317348 (+ 46us) server_negotiation.cc:407] Connection header received
0812 05:26:01.318019 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:01.318031 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:01.318113 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:01.318287 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:26:01.319027 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.320441 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.321301 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.321932 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.322043 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.325245 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:26:01.325249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:26:01.325250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:26:01.325262 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:26:01.327514 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:01.327991 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:01.327995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:01.327995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:01.328130 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:01.328363 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:01.328363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:01.328364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:01.328721 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:26:01.328880 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:01.329442 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 05:26:01.329618 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":120,"threads_started":1}
I20260812 05:26:01.335223 3339 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:26:01.335887 3339 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:26:01.340570 3339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89: Generated new TSK 0
I20260812 05:26:01.340834 3339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:26:01.378232 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40519
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:26:01.486148 3352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:26:01.486511 3352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:26:01.486547 3352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:26:01.487890 3352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:26:01.487939 3352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:26:01.487970 3352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:26:01.489560 3352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:26:01.489780 3352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:26:01.489984 3352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:26:01.496001 3359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:01.496008 3358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:01.495995 3361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:26:01.496035 3352 server_base.cc:1034] running on GCE node
I20260812 05:26:01.496451 3352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:26:01.496742 3352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:26:01.498009 3352 hybrid_clock.cc:648] HybridClock initialized: now 1786512361497980 us; error 45 us; skew 500 ppm
Aug 12 05:26:01 dist-test-slave-zpfg krb5kdc[3238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:26:01.501708 3352 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:26:01.502238 3352 webserver.cc:460] Webserver started at http://127.24.66.65:41169/ using document root <none> and password file <none>
I20260812 05:26:01.502544 3352 fs_manager.cc:359] Metadata directory not provided
I20260812 05:26:01.502602 3352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:26:01.502713 3352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:26:01.503952 3352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "0f10965c169e4530a0567dde23092541"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "073c0a591e973c8bffb6d4d8cc694f2e"
server_key_iv: "467f482ec67ce325b08f43924c5289be"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.504307 3352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "0f10965c169e4530a0567dde23092541"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "073c0a591e973c8bffb6d4d8cc694f2e"
server_key_iv: "467f482ec67ce325b08f43924c5289be"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.505991 3352 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 05:26:01.507055 3369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:01.507261 3352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:26:01.507377 3352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "0f10965c169e4530a0567dde23092541"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "073c0a591e973c8bffb6d4d8cc694f2e"
server_key_iv: "467f482ec67ce325b08f43924c5289be"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.507481 3352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:26:01.527283 3352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:26:01.529966 3352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:26:01.530149 3352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:26:01.530491 3352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:26:01.530856 3352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:26:01.530915 3352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:01.530953 3352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:26:01.530982 3352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:01.537634 3352 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:35485
I20260812 05:26:01.537706 3491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:35485 every 8 connection(s)
I20260812 05:26:01.538028 3352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
Aug 12 05:26:01 dist-test-slave-zpfg krb5kdc[3238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:26:01.546605 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 3352
I20260812 05:26:01.546725 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:26:01.546959 24841 external_mini_cluster.cc:1428] Setting key 2d16207334bd16a1d59cfef2e6436504
I20260812 05:26:01.548806 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.538923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:59235 (local address 127.24.66.126:40519)
0812 05:26:01.539058 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:01.539061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:01.539288 (+ 227us) server_negotiation.cc:407] Connection header received
0812 05:26:01.539887 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:01.539889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:01.539911 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:01.539969 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:26:01.541002 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.541547 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.542642 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.542871 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.542959 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.545347 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:26:01.545348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:26:01.545349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:26:01.545356 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:26:01.546856 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:01.547510 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:01.547512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:01.547512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:01.547578 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:01.548000 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:01.548001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:01.548001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:01.548120 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:26:01.548179 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:01.548638 (+ 459us) server_negotiation.cc:299] Negotiation successful
0812 05:26:01.548737 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260812 05:26:01.549136 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.538806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40519 (local address 127.24.66.65:59235)
0812 05:26:01.539086 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:01.539153 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:01.539767 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:01.540050 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:01.540054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:01.540674 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:26:01.540890 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.540896 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.541663 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.541664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.542543 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.542545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.542975 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.542976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.543103 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.543690 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:26:01.543699 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:26:01.545242 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:26:01.546968 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:01.546978 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:01.546981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:01.547423 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:01.547706 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:01.547707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:01.547708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:01.547905 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:01.548201 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:26:01.548209 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:26:01.548474 (+ 265us) client_negotiation.cc:769] Sending connection context
0812 05:26:01.548612 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 05:26:01.548818 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":128,"threads_started":1}
I20260812 05:26:01.549819 3492 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40519
I20260812 05:26:01.549952 3492 heartbeater.cc:464] Registering TS with master...
I20260812 05:26:01.550148 3492 heartbeater.cc:511] Master 127.24.66.126:40519 requested a full tablet report, sending...
I20260812 05:26:01.550809 3287 ts_manager.cc:194] Registered new tserver with Master: 0f10965c169e4530a0567dde23092541 (127.24.66.65:35485)
I20260812 05:26:01.551435 3287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:59235
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:26:01.581785 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40519
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:26:01.676103 3497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:26:01.676482 3497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:26:01.676539 3497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:26:01.678018 3497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:26:01.678107 3497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:26:01.678159 3497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:26:01.679942 3497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:26:01.680250 3497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:26:01.680534 3497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:26:01.686605 3502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:01.686599 3504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:01.686605 3506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:26:01.686636 3497 server_base.cc:1034] running on GCE node
I20260812 05:26:01.687139 3497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:26:01.687418 3497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:26:01.688588 3497 hybrid_clock.cc:648] HybridClock initialized: now 1786512361688564 us; error 42 us; skew 500 ppm
Aug 12 05:26:01 dist-test-slave-zpfg krb5kdc[3238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:26:01.691402 3497 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:26:01.691947 3497 webserver.cc:460] Webserver started at http://127.24.66.66:35463/ using document root <none> and password file <none>
I20260812 05:26:01.692245 3497 fs_manager.cc:359] Metadata directory not provided
I20260812 05:26:01.692298 3497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:26:01.692408 3497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:26:01.693540 3497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "a3a990152741439bb1882c9ef86f0697"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "eed347459b1f338927ae1bd3c779c5d1"
server_key_iv: "d4c4c066f7c5df693c5fcd8d192287a1"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.693892 3497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "a3a990152741439bb1882c9ef86f0697"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "eed347459b1f338927ae1bd3c779c5d1"
server_key_iv: "d4c4c066f7c5df693c5fcd8d192287a1"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.695514 3497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:26:01.696596 3515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:01.696914 3497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:26:01.697022 3497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "a3a990152741439bb1882c9ef86f0697"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "eed347459b1f338927ae1bd3c779c5d1"
server_key_iv: "d4c4c066f7c5df693c5fcd8d192287a1"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.697114 3497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:26:01.706499 3497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:26:01.709299 3497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:26:01.709494 3497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:26:01.709782 3497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:26:01.710136 3497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:26:01.710196 3497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:01.710245 3497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:26:01.710294 3497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:01.717374 3497 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:36073
I20260812 05:26:01.717537 3633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:36073 every 8 connection(s)
I20260812 05:26:01.717868 3497 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:26:01.718230 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 3497
I20260812 05:26:01.718366 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:26:01.718585 24841 external_mini_cluster.cc:1428] Setting key c4f96d6fb13519a30d8431f9ed53effb
Aug 12 05:26:01 dist-test-slave-zpfg krb5kdc[3238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
I20260812 05:26:01.730225 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.718755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:46801 (local address 127.24.66.126:40519)
0812 05:26:01.718810 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:01.718813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:01.719128 (+ 315us) server_negotiation.cc:407] Connection header received
0812 05:26:01.719799 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:01.719800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:01.719810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:01.719848 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:26:01.721311 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.721941 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.723138 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.723357 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.723438 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.726552 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:26:01.726553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:26:01.726554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:26:01.726566 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:26:01.728527 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:01.729073 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:01.729075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:01.729075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:01.729177 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:01.729474 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:01.729475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:01.729476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:01.729615 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:26:01.729701 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:01.730021 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 05:26:01.730120 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 05:26:01.730479 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.718743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40519 (local address 127.24.66.66:46801)
0812 05:26:01.718962 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:01.719017 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:01.719683 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:01.719941 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:01.719946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:01.720876 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:26:01.721189 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.721196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.722074 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.722076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.723028 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.723031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.723446 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.723447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.723579 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.724411 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:26:01.724421 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:26:01.726432 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:26:01.728661 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:01.728672 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:01.728674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:01.728979 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:01.729273 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:01.729274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:01.729274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:01.729393 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:01.729690 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:26:01.729697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:26:01.729864 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 05:26:01.729981 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 05:26:01.730173 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":100,"threads_started":1}
I20260812 05:26:01.731106 3634 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40519
I20260812 05:26:01.731231 3634 heartbeater.cc:464] Registering TS with master...
I20260812 05:26:01.731472 3634 heartbeater.cc:511] Master 127.24.66.126:40519 requested a full tablet report, sending...
I20260812 05:26:01.731930 3287 ts_manager.cc:194] Registered new tserver with Master: a3a990152741439bb1882c9ef86f0697 (127.24.66.66:36073)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
I20260812 05:26:01.732582 3287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:46801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:26:01.750464 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40519
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:26:01.839402 3639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:26:01.839622 3639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:26:01.839648 3639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:26:01.840910 3639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:26:01.840966 3639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:26:01.840996 3639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:26:01.842640 3639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:26:01.842866 3639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:26:01.843058 3639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:26:01.848428 3648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:01.848480 3645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:01.848428 3644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:26:01.848814 3639 server_base.cc:1034] running on GCE node
I20260812 05:26:01.849100 3639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:26:01.849388 3639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:26:01.850561 3639 hybrid_clock.cc:648] HybridClock initialized: now 1786512361850533 us; error 52 us; skew 500 ppm
Aug 12 05:26:01 dist-test-slave-zpfg krb5kdc[3238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:26:01.853246 3639 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:26:01.853760 3639 webserver.cc:460] Webserver started at http://127.24.66.67:39729/ using document root <none> and password file <none>
I20260812 05:26:01.853960 3639 fs_manager.cc:359] Metadata directory not provided
I20260812 05:26:01.854003 3639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:26:01.854099 3639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:26:01.855264 3639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "defca04602b740b0bf86db61c7740546"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "9a7764c48fd1db13ec9b0e6f52259e21"
server_key_iv: "0e996043d99c6e6c314810dad72598b4"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.855602 3639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "defca04602b740b0bf86db61c7740546"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "9a7764c48fd1db13ec9b0e6f52259e21"
server_key_iv: "0e996043d99c6e6c314810dad72598b4"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.856992 3639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:26:01.857889 3655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:01.858184 3639 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:26:01.858295 3639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "defca04602b740b0bf86db61c7740546"
format_stamp: "Formatted at 2026-08-12 05:26:01 on dist-test-slave-zpfg"
server_key: "9a7764c48fd1db13ec9b0e6f52259e21"
server_key_iv: "0e996043d99c6e6c314810dad72598b4"
server_key_version: "encryptionkey@0"
I20260812 05:26:01.858430 3639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:26:01.885710 3639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:26:01.888406 3639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:26:01.888537 3639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:26:01.888768 3639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:26:01.889071 3639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:26:01.889115 3639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:01.889142 3639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:26:01.889156 3639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:01.895941 3639 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:32797
I20260812 05:26:01.896025 3774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:32797 every 8 connection(s)
I20260812 05:26:01.896440 3639 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:26:01.897132 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 3639
I20260812 05:26:01.897269 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:26:01.897483 24841 external_mini_cluster.cc:1428] Setting key b05d4eeea5fbf139c6b12445780fb40b
Aug 12 05:26:01 dist-test-slave-zpfg krb5kdc[3238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:26:01.907087 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.897286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:52925 (local address 127.24.66.126:40519)
0812 05:26:01.897345 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:01.897349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:01.897629 (+ 280us) server_negotiation.cc:407] Connection header received
0812 05:26:01.898400 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:01.898401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:01.898413 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:01.898475 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:26:01.899572 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.900149 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.901246 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.901550 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.901629 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.904035 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:26:01.904037 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:26:01.904037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:26:01.904044 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:26:01.905672 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:01.906064 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:01.906065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:01.906065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:01.906122 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:01.906462 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:01.906463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:01.906468 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:01.906598 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:26:01.906665 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:01.906934 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 05:26:01.907008 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:26:01.907239 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.897255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40519 (local address 127.24.66.67:52925)
0812 05:26:01.897502 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:01.897547 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:01.898273 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:01.898571 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:01.898578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:01.899200 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:26:01.899404 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.899408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.900242 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.900242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.901167 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.901170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.901652 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.901653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.901780 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.902367 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:26:01.902376 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:26:01.903934 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:26:01.905756 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:01.905763 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:01.905765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:01.905995 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:01.906183 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:01.906184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:01.906184 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:01.906292 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:01.906665 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:26:01.906670 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:26:01.906798 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 05:26:01.906885 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 05:26:01.907015 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":163,"threads_started":1}
I20260812 05:26:01.907891 3775 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40519
I20260812 05:26:01.907982 3775 heartbeater.cc:464] Registering TS with master...
I20260812 05:26:01.908138 3775 heartbeater.cc:511] Master 127.24.66.126:40519 requested a full tablet report, sending...
I20260812 05:26:01.908515 3287 ts_manager.cc:194] Registered new tserver with Master: defca04602b740b0bf86db61c7740546 (127.24.66.67:32797)
I20260812 05:26:01.908972 3287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:52925
I20260812 05:26:01.909487 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:26:01.917965 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.910974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37246 (local address 127.24.66.126:40519)
0812 05:26:01.911026 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:01.911028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:01.911147 (+ 119us) server_negotiation.cc:407] Connection header received
0812 05:26:01.911201 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:01.911201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:01.911220 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:01.911268 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:26:01.911888 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.912590 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.913373 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.913575 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.913645 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.914995 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:26:01.914997 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:26:01.914997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:26:01.915005 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:26:01.916771 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:01.917208 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:01.917209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:01.917209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:01.917278 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:01.917474 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:01.917475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:01.917475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:01.917640 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:26:01.917725 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:01.917818 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 05:26:01.917889 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:26:01.920178 3287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:26:01.921954 3287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:26:01.929980 3787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.924714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:32797 (local address 127.0.0.1:57838)
0812 05:26:01.925196 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:01.925247 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:01.925325 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:01.925778 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:01.925795 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:01.925798 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:26:01.925856 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.925858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.927254 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.927255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.928545 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.928548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.929648 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.929649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.929737 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.929752 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 05:26:01.929826 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 05:26:01.929880 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":84,"threads_started":1}
I20260812 05:26:01.930215 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.924867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57838 (local address 127.24.66.67:32797)
0812 05:26:01.925507 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:01.925512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:01.925522 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:26:01.925565 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:01.925567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:01.925585 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:01.925663 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:26:01.925951 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.927134 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.928643 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.929547 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.929641 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.930045 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:01.930076 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 05:26:01.930121 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":149,"threads_started":1}
I20260812 05:26:01.930544 3789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.924963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:36073 (local address 127.0.0.1:53768)
0812 05:26:01.925339 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:01.925349 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:01.925392 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:01.925684 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:01.925687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:01.925689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:26:01.925828 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.925831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.928622 (+ 2791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.928622 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.929289 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.929290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.930257 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.930258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.930331 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.930359 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 05:26:01.930414 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 05:26:01.930455 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":103,"threads_started":1}
I20260812 05:26:01.930804 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.925123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53768 (local address 127.24.66.66:36073)
0812 05:26:01.925515 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:01.925520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:01.925531 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:26:01.925573 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:01.925575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:01.925592 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:01.925673 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:26:01.925924 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.927134 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.929389 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.930098 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.930171 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.930223 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:01.930665 (+ 442us) server_negotiation.cc:299] Negotiation successful
0812 05:26:01.930719 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":100,"threads_started":1}
I20260812 05:26:01.931308 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.924714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:35485 (local address 127.0.0.1:37628)
0812 05:26:01.925196 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:01.925247 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:01.925323 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:01.926919 (+ 1596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:01.926920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:01.926922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:26:01.926969 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.926970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.928447 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.928448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.929799 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:01.929802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.931041 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.931042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:01.931114 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.931119 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:26:01.931186 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:26:01.931232 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":98,"threads_started":1}
I20260812 05:26:01.931348 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:01.924932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37628 (local address 127.24.66.65:35485)
0812 05:26:01.926531 (+ 1599us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:01.926537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:01.926549 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:26:01.926635 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:01.926637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:01.926687 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:01.926809 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:26:01.927059 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.928253 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.929899 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:01.930926 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:01.931039 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:01.931098 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:01.931216 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 05:26:01.931260 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1521,"thread_start_us":141,"threads_started":1}
I20260812 05:26:01.931324 3705 tablet_service.cc:1467] Processing CreateTablet for tablet 3d22e098494c45c0b36fafd966aa78e2 (DEFAULT_TABLE table=test-table [id=0c3de4dc9fc74caaa0666f699477498d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:26:01.931743 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d22e098494c45c0b36fafd966aa78e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:26:01.932030 3421 tablet_service.cc:1467] Processing CreateTablet for tablet 3d22e098494c45c0b36fafd966aa78e2 (DEFAULT_TABLE table=test-table [id=0c3de4dc9fc74caaa0666f699477498d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:26:01.932380 3421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d22e098494c45c0b36fafd966aa78e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:26:01.934500 3792 tablet_bootstrap.cc:492] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546: Bootstrap starting.
I20260812 05:26:01.935320 3792 tablet_bootstrap.cc:654] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546: Neither blocks nor log segments found. Creating new log.
I20260812 05:26:01.935199 3563 tablet_service.cc:1467] Processing CreateTablet for tablet 3d22e098494c45c0b36fafd966aa78e2 (DEFAULT_TABLE table=test-table [id=0c3de4dc9fc74caaa0666f699477498d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:26:01.935506 3794 tablet_bootstrap.cc:492] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541: Bootstrap starting.
I20260812 05:26:01.935570 3563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d22e098494c45c0b36fafd966aa78e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:26:01.936118 3792 log.cc:824] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546: Log is configured to *not* fsync() on all Append() calls
I20260812 05:26:01.936184 3794 tablet_bootstrap.cc:654] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541: Neither blocks nor log segments found. Creating new log.
I20260812 05:26:01.936837 3794 log.cc:824] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541: Log is configured to *not* fsync() on all Append() calls
I20260812 05:26:01.937065 3792 tablet_bootstrap.cc:492] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546: No bootstrap required, opened a new log
I20260812 05:26:01.937170 3792 ts_tablet_manager.cc:1397] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:26:01.937731 3794 tablet_bootstrap.cc:492] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541: No bootstrap required, opened a new log
I20260812 05:26:01.937825 3794 ts_tablet_manager.cc:1397] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:26:01.939401 3792 raft_consensus.cc:348] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f10965c169e4530a0567dde23092541" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35485 } } peers { permanent_uuid: "defca04602b740b0bf86db61c7740546" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 32797 } } peers { permanent_uuid: "a3a990152741439bb1882c9ef86f0697" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36073 } }
I20260812 05:26:01.939582 3792 raft_consensus.cc:374] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:26:01.939800 3797 tablet_bootstrap.cc:492] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: Bootstrap starting.
I20260812 05:26:01.939628 3792 raft_consensus.cc:729] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: defca04602b740b0bf86db61c7740546, State: Initialized, Role: FOLLOWER
I20260812 05:26:01.940021 3792 consensus_queue.cc:260] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f10965c169e4530a0567dde23092541" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35485 } } peers { permanent_uuid: "defca04602b740b0bf86db61c7740546" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 32797 } } peers { permanent_uuid: "a3a990152741439bb1882c9ef86f0697" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36073 } }
I20260812 05:26:01.940003 3794 raft_consensus.cc:348] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f10965c169e4530a0567dde23092541" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35485 } } peers { permanent_uuid: "defca04602b740b0bf86db61c7740546" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 32797 } } peers { permanent_uuid: "a3a990152741439bb1882c9ef86f0697" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36073 } }
I20260812 05:26:01.940213 3794 raft_consensus.cc:374] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:26:01.940267 3794 raft_consensus.cc:729] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f10965c169e4530a0567dde23092541, State: Initialized, Role: FOLLOWER
I20260812 05:26:01.940343 3792 ts_tablet_manager.cc:1428] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:26:01.940410 3794 consensus_queue.cc:260] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f10965c169e4530a0567dde23092541" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35485 } } peers { permanent_uuid: "defca04602b740b0bf86db61c7740546" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 32797 } } peers { permanent_uuid: "a3a990152741439bb1882c9ef86f0697" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36073 } }
I20260812 05:26:01.940491 3775 heartbeater.cc:503] Master 127.24.66.126:40519 was elected leader, sending a full tablet report...
I20260812 05:26:01.940497 3797 tablet_bootstrap.cc:654] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: Neither blocks nor log segments found. Creating new log.
I20260812 05:26:01.940707 3794 ts_tablet_manager.cc:1428] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:26:01.940826 3492 heartbeater.cc:503] Master 127.24.66.126:40519 was elected leader, sending a full tablet report...
I20260812 05:26:01.941141 3797 log.cc:824] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: Log is configured to *not* fsync() on all Append() calls
I20260812 05:26:01.942059 3797 tablet_bootstrap.cc:492] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: No bootstrap required, opened a new log
I20260812 05:26:01.942171 3797 ts_tablet_manager.cc:1397] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:26:01.943884 3797 raft_consensus.cc:348] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f10965c169e4530a0567dde23092541" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35485 } } peers { permanent_uuid: "defca04602b740b0bf86db61c7740546" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 32797 } } peers { permanent_uuid: "a3a990152741439bb1882c9ef86f0697" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36073 } }
I20260812 05:26:01.944106 3797 raft_consensus.cc:374] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:26:01.944142 3797 raft_consensus.cc:729] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a990152741439bb1882c9ef86f0697, State: Initialized, Role: FOLLOWER
I20260812 05:26:01.944303 3797 consensus_queue.cc:260] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f10965c169e4530a0567dde23092541" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35485 } } peers { permanent_uuid: "defca04602b740b0bf86db61c7740546" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 32797 } } peers { permanent_uuid: "a3a990152741439bb1882c9ef86f0697" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36073 } }
I20260812 05:26:01.944670 3797 ts_tablet_manager.cc:1428] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 05:26:01.944703 3634 heartbeater.cc:503] Master 127.24.66.126:40519 was elected leader, sending a full tablet report...
W20260812 05:26:01.968726 3635 tablet.cc:2369] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:26:02.039011 3493 tablet.cc:2369] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:26:02.069968 3801 raft_consensus.cc:482] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:26:02.070212 3801 raft_consensus.cc:504] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f10965c169e4530a0567dde23092541" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35485 } } peers { permanent_uuid: "defca04602b740b0bf86db61c7740546" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 32797 } } peers { permanent_uuid: "a3a990152741439bb1882c9ef86f0697" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36073 } }
I20260812 05:26:02.070727 3801 leader_election.cc:288] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f10965c169e4530a0567dde23092541 (127.24.66.65:35485), defca04602b740b0bf86db61c7740546 (127.24.66.67:32797)
I20260812 05:26:02.074970 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:02.071041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:60007 (local address 127.24.66.65:35485)
0812 05:26:02.071103 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:02.071106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:02.071119 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:26:02.071158 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:02.071159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:02.071172 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:02.071285 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:26:02.071508 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:02.072352 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:02.073973 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:02.074591 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:02.074675 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:02.074685 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:02.074841 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 05:26:02.074891 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:26:02.075034 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:02.070953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:32797 (local address 127.24.66.66:57597)
0812 05:26:02.071148 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:02.071157 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:02.071211 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:02.071344 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:02.071345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:02.071346 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:26:02.071402 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:02.071404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:02.072412 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:02.072414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:02.073834 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:02.073839 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:02.074728 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:02.074731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:02.074825 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:02.074831 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:26:02.074888 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 05:26:02.074931 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":89,"thread_start_us":108,"threads_started":1}
I20260812 05:26:02.075030 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:02.071079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:57597 (local address 127.24.66.67:32797)
0812 05:26:02.071148 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:02.071150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:02.071232 (+ 82us) server_negotiation.cc:407] Connection header received
0812 05:26:02.071262 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:02.071263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:02.071273 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:02.071332 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:26:02.071504 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:02.072270 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:02.073973 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:02.074591 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:02.074676 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:02.074685 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:02.074913 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 05:26:02.074965 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:26:02.075162 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:02.070953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:35485 (local address 127.24.66.66:60007)
0812 05:26:02.071029 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:02.071043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:02.071097 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:02.071302 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:02.071303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:02.071304 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:26:02.071402 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:02.071404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:02.072429 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:02.072430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:02.073860 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:02.073862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:02.074683 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:02.074684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:02.074761 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:02.074765 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:26:02.074819 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 05:26:02.074857 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":93}
I20260812 05:26:02.075462 3725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d22e098494c45c0b36fafd966aa78e2" candidate_uuid: "a3a990152741439bb1882c9ef86f0697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "defca04602b740b0bf86db61c7740546" is_pre_election: true
I20260812 05:26:02.075496 3443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d22e098494c45c0b36fafd966aa78e2" candidate_uuid: "a3a990152741439bb1882c9ef86f0697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f10965c169e4530a0567dde23092541" is_pre_election: true
I20260812 05:26:02.075635 3725 raft_consensus.cc:2454] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3a990152741439bb1882c9ef86f0697 in term 0.
I20260812 05:26:02.075726 3443 raft_consensus.cc:2454] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3a990152741439bb1882c9ef86f0697 in term 0.
I20260812 05:26:02.075906 3517 leader_election.cc:302] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3a990152741439bb1882c9ef86f0697, defca04602b740b0bf86db61c7740546; no voters:
I20260812 05:26:02.076045 3801 raft_consensus.cc:2781] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:26:02.076217 3801 raft_consensus.cc:482] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:26:02.076251 3801 raft_consensus.cc:3037] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:26:02.077010 3801 raft_consensus.cc:504] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f10965c169e4530a0567dde23092541" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35485 } } peers { permanent_uuid: "defca04602b740b0bf86db61c7740546" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 32797 } } peers { permanent_uuid: "a3a990152741439bb1882c9ef86f0697" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36073 } }
I20260812 05:26:02.077175 3801 leader_election.cc:288] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [CANDIDATE]: Term 1 election: Requested vote from peers 0f10965c169e4530a0567dde23092541 (127.24.66.65:35485), defca04602b740b0bf86db61c7740546 (127.24.66.67:32797)
I20260812 05:26:02.077370 3725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d22e098494c45c0b36fafd966aa78e2" candidate_uuid: "a3a990152741439bb1882c9ef86f0697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "defca04602b740b0bf86db61c7740546"
I20260812 05:26:02.077399 3443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d22e098494c45c0b36fafd966aa78e2" candidate_uuid: "a3a990152741439bb1882c9ef86f0697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f10965c169e4530a0567dde23092541"
I20260812 05:26:02.077469 3725 raft_consensus.cc:3037] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:26:02.077471 3443 raft_consensus.cc:3037] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:26:02.078478 3725 raft_consensus.cc:2454] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a990152741439bb1882c9ef86f0697 in term 1.
I20260812 05:26:02.078478 3443 raft_consensus.cc:2454] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a990152741439bb1882c9ef86f0697 in term 1.
I20260812 05:26:02.078706 3517 leader_election.cc:302] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3a990152741439bb1882c9ef86f0697, defca04602b740b0bf86db61c7740546; no voters:
I20260812 05:26:02.078807 3801 raft_consensus.cc:2781] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:26:02.078989 3801 raft_consensus.cc:686] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 1 LEADER]: Becoming Leader. State: Replica: a3a990152741439bb1882c9ef86f0697, State: Running, Role: LEADER
I20260812 05:26:02.079110 3801 consensus_queue.cc:237] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f10965c169e4530a0567dde23092541" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35485 } } peers { permanent_uuid: "defca04602b740b0bf86db61c7740546" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 32797 } } peers { permanent_uuid: "a3a990152741439bb1882c9ef86f0697" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36073 } }
I20260812 05:26:02.080049 3286 catalog_manager.cc:5495] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3a990152741439bb1882c9ef86f0697 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "a3a990152741439bb1882c9ef86f0697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f10965c169e4530a0567dde23092541" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "defca04602b740b0bf86db61c7740546" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 32797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3a990152741439bb1882c9ef86f0697" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36073 } health_report { overall_health: HEALTHY } } }
I20260812 05:26:02.116324 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:02.113708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53782 (local address 127.24.66.66:36073)
0812 05:26:02.113754 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:02.113757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:02.113766 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:26:02.113839 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:02.113841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:02.113876 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:02.113963 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:26:02.114114 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:02.114621 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:02.115443 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:02.115642 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:02.115737 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:02.115926 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:26:02.116038 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:26:02.116150 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:02.116169 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 05:26:02.116192 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:26:02.118600 3725 raft_consensus.cc:1261] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546 [term 1 FOLLOWER]: Refusing update from remote peer a3a990152741439bb1882c9ef86f0697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:26:02.118600 3443 raft_consensus.cc:1261] T 3d22e098494c45c0b36fafd966aa78e2 P 0f10965c169e4530a0567dde23092541 [term 1 FOLLOWER]: Refusing update from remote peer a3a990152741439bb1882c9ef86f0697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:26:02.119071 3803 consensus_queue.cc:1035] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "defca04602b740b0bf86db61c7740546" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 32797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:26:02.119346 3801 consensus_queue.cc:1035] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f10965c169e4530a0567dde23092541" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 35485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:26:02.121939 3807 mvcc.cc:205] Tried to move back new op lower bound from 7317554635234152448 to 7317554635076771840. Current Snapshot: MvccSnapshot[applied={T|T < 7317554635234152448}]
I20260812 05:26:02.122331 3809 mvcc.cc:205] Tried to move back new op lower bound from 7317554635234152448 to 7317554635076771840. Current Snapshot: MvccSnapshot[applied={T|T < 7317554635234152448}]
I20260812 05:26:02.124668 3287 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:26:02.124822 3287 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:26:02.125860 3287 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 212cdedf9461450ea65a2ec9a9d42d89: Sending DeleteTablet for 3 replicas of tablet 3d22e098494c45c0b36fafd966aa78e2
I20260812 05:26:02.126252 3421 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d22e098494c45c0b36fafd966aa78e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:26:02 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:37628
I20260812 05:26:02.126398 3705 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d22e098494c45c0b36fafd966aa78e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:26:02 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:57838
I20260812 05:26:02.126652 3563 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d22e098494c45c0b36fafd966aa78e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:26:02 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:53768
I20260812 05:26:02.126749 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 3352
I20260812 05:26:02.129428 3819 tablet_replica.cc:330] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: stopping tablet replica
I20260812 05:26:02.129598 3819 raft_consensus.cc:2229] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:26:02.129806 3819 raft_consensus.cc:2258] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:26:02.130044 3818 tablet_replica.cc:330] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546: stopping tablet replica
I20260812 05:26:02.130179 3818 raft_consensus.cc:2229] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:26:02.130306 3818 raft_consensus.cc:2258] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:26:02.130965 3819 ts_tablet_manager.cc:1905] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:26:02.130965 3818 ts_tablet_manager.cc:1905] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:26:02.133167 3819 ts_tablet_manager.cc:1918] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:26:02.133271 3818 ts_tablet_manager.cc:1918] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:26:02.133358 3819 log.cc:1196] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/3d22e098494c45c0b36fafd966aa78e2
I20260812 05:26:02.133394 3818 log.cc:1196] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/3d22e098494c45c0b36fafd966aa78e2
I20260812 05:26:02.133744 3819 ts_tablet_manager.cc:1939] T 3d22e098494c45c0b36fafd966aa78e2 P a3a990152741439bb1882c9ef86f0697: Deleting consensus metadata
I20260812 05:26:02.133853 3818 ts_tablet_manager.cc:1939] T 3d22e098494c45c0b36fafd966aa78e2 P defca04602b740b0bf86db61c7740546: Deleting consensus metadata
I20260812 05:26:02.134305 3270 catalog_manager.cc:4841] TS defca04602b740b0bf86db61c7740546 (127.24.66.67:32797): tablet 3d22e098494c45c0b36fafd966aa78e2 (table test-table [id=0c3de4dc9fc74caaa0666f699477498d]) successfully deleted
I20260812 05:26:02.134464 3270 catalog_manager.cc:4841] TS a3a990152741439bb1882c9ef86f0697 (127.24.66.66:36073): tablet 3d22e098494c45c0b36fafd966aa78e2 (table test-table [id=0c3de4dc9fc74caaa0666f699477498d]) successfully deleted
W20260812 05:26:02.143464 3269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:35485 (error 108)
I20260812 05:26:02.144155 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:02.143989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:35485 (local address 127.0.0.1:37636)
0812 05:26:02.144057 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:02.144112 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:35485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 05:26:02.144244 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 3497
W20260812 05:26:02.147516 3269 catalog_manager.cc:4584] TS 0f10965c169e4530a0567dde23092541 (127.24.66.65:35485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d22e098494c45c0b36fafd966aa78e2: Network error: Client connection negotiation failed: client connection to 127.24.66.65:35485: connect: Connection refused (error 111)
I20260812 05:26:02.151640 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 3639
I20260812 05:26:02.157843 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 3252
2026-08-12T05:26:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2246 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:26:02 dist-test-slave-zpfg krb5kdc[3823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:26:02 dist-test-slave-zpfg krb5kdc[3823](info): set up 2 sockets
Aug 12 05:26:02 dist-test-slave-zpfg krb5kdc[3823](info): commencing operation
krb5kdc: starting...
W20260812 05:26:03.308254 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:26:03 dist-test-slave-zpfg krb5kdc[3823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:26:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:26:03Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.66.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.66.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/rangeradmin_127.24.66.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/rangeradmin_127.24.66.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.66.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.66.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/rangerlookup_127.24.66.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/rangerlookup_127.24.66.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.66.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.66.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/HTTP_127.24.66.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/HTTP_127.24.66.84@KRBTEST.COM.keytab.
I20260812 05:26:03.505576 24841 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task_bqoao/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/postgres -l logfile start
2026-08-12 05:26:06.377 UTC [3854] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 05:26:06.377 UTC [3854] LOG: listening on IPv4 address "127.24.66.84", port 40223
2026-08-12 05:26:06.382 UTC [3854] LOG: listening on Unix socket "/tmp/.s.PGSQL.40223"
2026-08-12 05:26:06.406 UTC [3857] LOG: database system was shut down at 2026-08-12 05:26:04 UTC
2026-08-12 05:26:06.410 UTC [3854] LOG: database system is ready to accept connections
I20260812 05:26:07.492925 24841 mini_postgres.cc:96] Postgres bound to 40223
2026-08-12 05:26:07.499 UTC [3865] FATAL: database "slave" does not exist
127.24.66.84:40223 - accepting connections
I20260812 05:26:07.500597 24841 mini_ranger.cc:161] Starting Ranger...
I20260812 05:26:07.517263 24841 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 05:26:08.072528 24841 mini_ranger.cc:87] Created ranger Postgres database
I20260812 05:26:08.072633 24841 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-admin
2026-08-12 05:26:08,395 [I] DB FLAVOR :POSTGRES
2026-08-12 05:26:08,396 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 05:26:08,396 [I] Checking connection..
2026-08-12 05:26:08,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 05:26:08,694 [I] Checking connection passed.
2026-08-12 05:26:08,694 [I] --------- Verifying version history table ---------
2026-08-12 05:26:08,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:26:08,974 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 05:26:08,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:26:09,256 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 05:26:09,256 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 05:26:09,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 05:26:09.515 UTC [3932] WARNING: there is no transaction in progress
2026-08-12 05:26:09,523 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 05:26:09,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:26:09,797 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 05:26:09,798 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 05:26:09,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 05:26:10,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 05:26:10,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zpfg.c.gcp-upstream.internal','N') ;"
2026-08-12 05:26:10,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 05:26:10,895 [I] Table x_portal_user does not exist in database ranger
2026-08-12 05:26:10,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 05:26:11,141 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 05:26:11,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 05:26:11,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 05:26:11,639 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 05:26:11,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 05:26:12.716 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.736 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.755 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.772 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.793 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.813 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.878 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.886 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.904 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.915 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.927 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.942 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.953 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.960 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.968 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:12.976 UTC [4046] WARNING: there is no transaction in progress
2026-08-12 05:26:13,778 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 05:26:13,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 05:26:14,058 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 05:26:14,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 05:26:14,343 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 05:26:14,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 05:26:14,632 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 05:26:14,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 05:26:14,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 05:26:15,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zpfg.c.gcp-upstream.internal';"
2026-08-12 05:26:15,414 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 05:26:15,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 05:26:15,663 [I] DB_PATCHES have already been applied
I20260812 05:26:15.669309 24841 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-admin:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task_bqoao/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 05:26:15.669418 24841 mini_ranger.cc:193] Using host: 127.24.66.84
I20260812 05:26:15.673607 24841 mini_ranger.cc:227] Ranger admin URL: http://127.24.66.84:35153
Aug 12, 2026 5:26:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:26:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:26:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 5:26:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 5:26:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 5:26:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 5:26:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 05:26:16.912613 24841 mini_ranger.cc:160] Time spent starting Ranger: real 9.412s user 0.000s sys 0.013s
Aug 12, 2026 5:26:40 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 5:26:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 12, 2026 5:26:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 5:26:40 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 5:26:41 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 05:26:42.307978 24841 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.66.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.66.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/rangerkms_127.24.66.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/rangerkms_127.24.66.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.66.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.66.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.66.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/HTTP_127.24.66.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.66.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/HTTP_127.24.66.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 05:26:42 dist-test-slave-zpfg krb5kdc[3823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512402, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 05:26:42.412431 24841 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 05:26:42.427850 24841 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 05:26:43.476444 24841 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 05:26:43.476539 24841 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-kms
I20260812 05:26:43.476558 24841 mini_ranger_kms.cc:227] Using postgres at 127.24.66.84:40223
2026-08-12 05:26:43,968 [I] DB FLAVOR :POSTGRES
2026-08-12 05:26:43,969 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 05:26:43,969 [I] Checking connection
2026-08-12 05:26:43,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 05:26:44,276 [I] connection success
2026-08-12 05:26:44,276 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 05:26:44,276 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 05:26:44,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 05:26:44,606 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 05:26:44,606 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 05:26:44,606 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 05:26:44,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:40223/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 05:26:44,967 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 05:26:45.512655 24841 mini_ranger_kms.cc:310] Created kms service
I20260812 05:26:45.708614 24841 mini_ranger_kms.cc:326] Created kudu user
I20260812 05:26:45.763319 24841 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 05:26:45 dist-test-slave-zpfg krb5kdc[3823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512402, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:26:46.210474 24841 mini_ranger_kms.cc:384] Added ranger policy
I20260812 05:26:46.210595 24841 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-kms:/tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 05:26:46.210630 24841 mini_ranger_kms.cc:240] Using host: 127.24.66.84
I20260812 05:26:46.214869 24841 mini_ranger_kms.cc:276] Ranger KMS PID: 4237
I20260812 05:26:46.215015 24841 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.66.84:54233
Aug 12, 2026 5:26:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:26:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:26:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 5:26:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 5:26:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 5:26:48 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 05:26:48.688198 24841 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.276s user 0.017s sys 0.443s
I20260812 05:26:48.688310 24841 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 05:26:48.688385 24841 mini_ranger_kms.cc:401] 127.24.66.84:54233/kms/v1/keys
Aug 12 05:26:48 dist-test-slave-zpfg krb5kdc[3823](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786512408, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.66.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 05:26:49 dist-test-slave-zpfg krb5kdc[3823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512408, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.66.84@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
Aug 12 05:26:49 dist-test-slave-zpfg krb5kdc[3823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512408, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.66.84@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
Aug 12 05:26:52 dist-test-slave-zpfg krb5kdc[3823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:26:52.455024 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:36219
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:36219
--ranger_config_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:54233/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:26:52.555063 4288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:26:52.555378 4288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:26:52.555428 4288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:26:52.556851 4288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:26:52.556938 4288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:26:52.556967 4288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:26:52.556999 4288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:26:52.557044 4288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:26:52.559067 4288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.66.84:54233/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:36219
--ranger_config_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:36219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:26:52.559412 4288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:26:52.559701 4288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:26:52.566666 4294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:26:52.566835 4288 server_base.cc:1034] running on GCE node
W20260812 05:26:52.566730 4296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:52.566666 4293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:26:52.567371 4288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:26:52.567706 4288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:26:52.568890 4288 hybrid_clock.cc:648] HybridClock initialized: now 1786512412568868 us; error 42 us; skew 500 ppm
Aug 12 05:26:52 dist-test-slave-zpfg krb5kdc[3823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512412, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:26:52.577988 4288 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:26:52.578742 4288 webserver.cc:460] Webserver started at http://127.24.66.126:39997/ using document root <none> and password file <none>
I20260812 05:26:52.579012 4288 fs_manager.cc:359] Metadata directory not provided
I20260812 05:26:52.579078 4288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:26:52.579450 4288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:26:52 dist-test-slave-zpfg krb5kdc[3823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512412, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:26:52.649703 4288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "3b92e6d12a9e4a61aeb99851553b1ec6"
format_stamp: "Formatted at 2026-08-12 05:26:52 on dist-test-slave-zpfg"
server_key: "6f01a2707e4c83faf8fe331837df4cdf"
server_key_iv: "9441b6921ad5be4dba2eaba0a39c16ec"
server_key_version: "kuduclusterkey@0"
I20260812 05:26:52.650249 4288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "3b92e6d12a9e4a61aeb99851553b1ec6"
format_stamp: "Formatted at 2026-08-12 05:26:52 on dist-test-slave-zpfg"
server_key: "6f01a2707e4c83faf8fe331837df4cdf"
server_key_iv: "9441b6921ad5be4dba2eaba0a39c16ec"
server_key_version: "kuduclusterkey@0"
I20260812 05:26:52.652522 4288 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 05:26:52.665952 4303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:52.666204 4288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:26:52.666319 4288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "3b92e6d12a9e4a61aeb99851553b1ec6"
format_stamp: "Formatted at 2026-08-12 05:26:52 on dist-test-slave-zpfg"
server_key: "6f01a2707e4c83faf8fe331837df4cdf"
server_key_iv: "9441b6921ad5be4dba2eaba0a39c16ec"
server_key_version: "kuduclusterkey@0"
I20260812 05:26:52.666527 4288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:26:52.690585 4288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:26:52.693039 4288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:26:52.693228 4288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:26:52.699607 4288 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:36219
I20260812 05:26:52.699632 4358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:36219 every 8 connection(s)
I20260812 05:26:52.700150 4288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:26:52.701011 4359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:26:52.703501 4359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6: Bootstrap starting.
I20260812 05:26:52.703820 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 4288
I20260812 05:26:52.703965 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:26:52.704200 24841 external_mini_cluster.cc:1428] Setting key 452b885a5466a9d0d2d419321df566f5
I20260812 05:26:52.704371 4359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6: Neither blocks nor log segments found. Creating new log.
I20260812 05:26:52.704761 4359 log.cc:824] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6: Log is configured to *not* fsync() on all Append() calls
I20260812 05:26:52.705727 4359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6: No bootstrap required, opened a new log
I20260812 05:26:52.708500 4359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b92e6d12a9e4a61aeb99851553b1ec6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36219 } }
I20260812 05:26:52.708688 4359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:26:52.708755 4359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b92e6d12a9e4a61aeb99851553b1ec6, State: Initialized, Role: FOLLOWER
I20260812 05:26:52.708902 4359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b92e6d12a9e4a61aeb99851553b1ec6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36219 } }
I20260812 05:26:52.709005 4359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:26:52.709061 4359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:26:52.709137 4359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:26:52.709869 4359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b92e6d12a9e4a61aeb99851553b1ec6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36219 } }
I20260812 05:26:52.709983 4359 leader_election.cc:302] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b92e6d12a9e4a61aeb99851553b1ec6; no voters:
I20260812 05:26:52.710201 4359 leader_election.cc:288] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:26:52.710423 4364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:26:52.710623 4359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:26:52.710623 4364 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [term 1 LEADER]: Becoming Leader. State: Replica: 3b92e6d12a9e4a61aeb99851553b1ec6, State: Running, Role: LEADER
I20260812 05:26:52.710841 4364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b92e6d12a9e4a61aeb99851553b1ec6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36219 } }
I20260812 05:26:52.711398 4365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b92e6d12a9e4a61aeb99851553b1ec6. Latest consensus state: current_term: 1 leader_uuid: "3b92e6d12a9e4a61aeb99851553b1ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b92e6d12a9e4a61aeb99851553b1ec6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36219 } } }
I20260812 05:26:52.711499 4365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [sys.catalog]: This master's current role is: LEADER
I20260812 05:26:52.711784 4367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b92e6d12a9e4a61aeb99851553b1ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b92e6d12a9e4a61aeb99851553b1ec6" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36219 } } }
I20260812 05:26:52.711925 4367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6 [sys.catalog]: This master's current role is: LEADER
I20260812 05:26:52.712204 4369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 05:26:52 dist-test-slave-zpfg krb5kdc[3823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:26:52.713028 4369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:26:52.713944 4359 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 05:26:52.716354 4369 catalog_manager.cc:1347] Generated new cluster ID: 3faa1b25eaff4cefbc97b9550c537e23
I20260812 05:26:52.716471 4369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:26:52.717938 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:52.704768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41044 (local address 127.24.66.126:36219)
0812 05:26:52.704987 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:52.704998 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:52.705035 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:26:52.705616 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:52.705625 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:52.705690 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:52.706737 (+ 1047us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:26:52.706836 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:52.708352 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:52.709447 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:52.710130 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:52.710247 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:52.713264 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:26:52.713268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:26:52.713269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:26:52.713282 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:26:52.715464 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:52.715898 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:52.715902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:52.715902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:52.716016 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:52.716322 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:52.716323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:52.716323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:52.716690 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:26:52.716854 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:52.717498 (+ 644us) server_negotiation.cc:299] Negotiation successful
0812 05:26:52.717679 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":97,"threads_started":1}
I20260812 05:26:52.754834 4369 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:26:52.755671 4369 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:26:52.760178 4369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6: Generated new TSK 0
I20260812 05:26:52.760622 4369 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:26:53.533936 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:54233/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:36219
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:26:53.649695 4408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:26:53.650234 4408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:26:53.650336 4408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:26:53.651782 4408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:26:53.651859 4408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:26:53.651908 4408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:26:53.653590 4408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.66.84:54233/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:36219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:26:53.653872 4408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:26:53.654168 4408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:26:53.659713 4416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:53.659777 4413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:53.659780 4414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:26:53.660038 4408 server_base.cc:1034] running on GCE node
I20260812 05:26:53.660348 4408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:26:53.660559 4408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:26:53.661741 4408 hybrid_clock.cc:648] HybridClock initialized: now 1786512413661713 us; error 42 us; skew 500 ppm
Aug 12 05:26:53 dist-test-slave-zpfg krb5kdc[3823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512413, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:26:53.665313 4408 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:26:53.665902 4408 webserver.cc:460] Webserver started at http://127.24.66.65:35693/ using document root <none> and password file <none>
I20260812 05:26:53.666208 4408 fs_manager.cc:359] Metadata directory not provided
I20260812 05:26:53.666290 4408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:26:53.666471 4408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:26:53 dist-test-slave-zpfg krb5kdc[3823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512413, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:26:53.683105 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "e90a8005e4fc4aef8471f4f1e80b860c"
format_stamp: "Formatted at 2026-08-12 05:26:53 on dist-test-slave-zpfg"
server_key: "c1d7885dd53fd6c8db5652b3f48d8dc1"
server_key_iv: "b8548678e679c2e08a2833001ce6c46d"
server_key_version: "kuduclusterkey@0"
I20260812 05:26:53.683796 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "e90a8005e4fc4aef8471f4f1e80b860c"
format_stamp: "Formatted at 2026-08-12 05:26:53 on dist-test-slave-zpfg"
server_key: "c1d7885dd53fd6c8db5652b3f48d8dc1"
server_key_iv: "b8548678e679c2e08a2833001ce6c46d"
server_key_version: "kuduclusterkey@0"
I20260812 05:26:53.686221 4408 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 05:26:53.704515 4424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:53.704849 4408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:26:53.704977 4408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "e90a8005e4fc4aef8471f4f1e80b860c"
format_stamp: "Formatted at 2026-08-12 05:26:53 on dist-test-slave-zpfg"
server_key: "c1d7885dd53fd6c8db5652b3f48d8dc1"
server_key_iv: "b8548678e679c2e08a2833001ce6c46d"
server_key_version: "kuduclusterkey@0"
I20260812 05:26:53.705103 4408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:26:53.722380 4408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:26:53.725292 4408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:26:53.725538 4408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:26:53.725858 4408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:26:53.726329 4408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:26:53.726460 4408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:53.726573 4408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:26:53.726668 4408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:53.733888 4408 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:41539
I20260812 05:26:53.735257 4408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:26:53.736380 4546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:41539 every 8 connection(s)
Aug 12 05:26:53 dist-test-slave-zpfg krb5kdc[3823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512413, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:26:53.743178 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 4408
I20260812 05:26:53.743299 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:26:53.743527 24841 external_mini_cluster.cc:1428] Setting key ebfda277ff15fce2f17c7899dea7a7eb
I20260812 05:26:53.747605 4549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:53.736147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:36219 (local address 127.24.66.65:39427)
0812 05:26:53.736832 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:53.736881 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:53.737426 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:53.737737 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:53.737741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:53.737940 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:26:53.738156 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:53.738160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:53.739049 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:53.739051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:53.740372 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:53.740376 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:53.740974 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:53.740975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:53.741133 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:53.741831 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:26:53.741840 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:26:53.743291 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:26:53.745654 (+ 2363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:53.745663 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:53.745666 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:53.746028 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:53.746267 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:53.746268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:53.746268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:53.746466 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:53.746803 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:26:53.746810 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:26:53.747001 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 05:26:53.747133 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 05:26:53.747331 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":106,"threads_started":1}
I20260812 05:26:53.748137 4550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:53.736536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:39427 (local address 127.24.66.126:36219)
0812 05:26:53.736756 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:53.736759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:53.736997 (+ 238us) server_negotiation.cc:407] Connection header received
0812 05:26:53.737545 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:53.737547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:53.737564 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:53.737654 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:26:53.738257 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:53.738932 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:53.740501 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:53.740817 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:53.740889 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:53.743440 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:26:53.743442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:26:53.743442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:26:53.743452 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:26:53.745554 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:53.746125 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:53.746126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:53.746127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:53.746195 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:53.746545 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:53.746546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:53.746547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:53.746712 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:26:53.746774 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:53.747891 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0812 05:26:53.747986 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":100,"threads_started":1}
I20260812 05:26:53.748543 4547 heartbeater.cc:347] Connected to a master server at 127.24.66.126:36219
I20260812 05:26:53.748649 4547 heartbeater.cc:464] Registering TS with master...
I20260812 05:26:53.748862 4547 heartbeater.cc:511] Master 127.24.66.126:36219 requested a full tablet report, sending...
I20260812 05:26:53.749462 4314 ts_manager.cc:194] Registered new tserver with Master: e90a8005e4fc4aef8471f4f1e80b860c (127.24.66.65:41539)
I20260812 05:26:53.750113 4314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:39427
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:26:53.785218 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:54233/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:36219
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:26:53.897823 4553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:26:53.898129 4553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:26:53.898187 4553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:26:53.899582 4553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:26:53.899667 4553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:26:53.899719 4553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:26:53.901362 4553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.66.84:54233/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:36219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:26:53.901639 4553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:26:53.901909 4553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:26:53.907644 4563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:53.907960 4560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:26:53.908295 4553 server_base.cc:1034] running on GCE node
W20260812 05:26:53.908479 4561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:26:53.908717 4553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:26:53.909094 4553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:26:53.910264 4553 hybrid_clock.cc:648] HybridClock initialized: now 1786512413910249 us; error 31 us; skew 500 ppm
Aug 12 05:26:53 dist-test-slave-zpfg krb5kdc[3823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512413, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:26:53.913507 4553 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:26:53.914067 4553 webserver.cc:460] Webserver started at http://127.24.66.66:44323/ using document root <none> and password file <none>
I20260812 05:26:53.914387 4553 fs_manager.cc:359] Metadata directory not provided
I20260812 05:26:53.914455 4553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:26:53.914595 4553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:26:53 dist-test-slave-zpfg krb5kdc[3823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512413, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:26:53.931262 4553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878"
format_stamp: "Formatted at 2026-08-12 05:26:53 on dist-test-slave-zpfg"
server_key: "dccd8d6b8d8f0876a2a6121ad041cdfa"
server_key_iv: "c981421ec635d2862540504c978e47bc"
server_key_version: "kuduclusterkey@0"
I20260812 05:26:53.931810 4553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878"
format_stamp: "Formatted at 2026-08-12 05:26:53 on dist-test-slave-zpfg"
server_key: "dccd8d6b8d8f0876a2a6121ad041cdfa"
server_key_iv: "c981421ec635d2862540504c978e47bc"
server_key_version: "kuduclusterkey@0"
Aug 12 05:26:53 dist-test-slave-zpfg krb5kdc[3823](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786512413, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:26:53.933908 4553 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 05:26:53.954507 4571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:53.954787 4553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260812 05:26:53.954900 4553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878"
format_stamp: "Formatted at 2026-08-12 05:26:53 on dist-test-slave-zpfg"
server_key: "dccd8d6b8d8f0876a2a6121ad041cdfa"
server_key_iv: "c981421ec635d2862540504c978e47bc"
server_key_version: "kuduclusterkey@0"
I20260812 05:26:53.955003 4553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:26:53.979653 4553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:26:53.982746 4553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:26:53.983055 4553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:26:53.983407 4553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:26:53.983878 4553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:26:53.983978 4553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:53.984112 4553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:26:53.984184 4553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:53.994488 4553 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:41659
I20260812 05:26:53.995165 4553 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:26:54.003417 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 4553
I20260812 05:26:54.003577 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:26:54.003821 24841 external_mini_cluster.cc:1428] Setting key f6e7a741a7a5225c888c3830fa6be7d0
I20260812 05:26:54.006464 4689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:41659 every 8 connection(s)
Aug 12 05:26:54 dist-test-slave-zpfg krb5kdc[3823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512413, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
I20260812 05:26:54.021811 4550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.007871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:37839 (local address 127.24.66.126:36219)
0812 05:26:54.007925 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:54.007927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:54.008305 (+ 378us) server_negotiation.cc:407] Connection header received
0812 05:26:54.009517 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:54.009518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:54.009537 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:54.009609 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:26:54.010989 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.011899 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.013706 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.014108 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.014160 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.018006 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:26:54.018008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:26:54.018009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:26:54.018017 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:26:54.019995 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:54.020540 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:54.020542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:54.020542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:54.020622 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:54.020967 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:54.020968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:54.020969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:54.021130 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:26:54.021205 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:54.021588 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 05:26:54.021709 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:26:54.022832 4693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.007672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:36219 (local address 127.24.66.66:37839)
0812 05:26:54.008142 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:54.008188 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:54.009384 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:54.009781 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:54.009787 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:54.010377 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:26:54.010848 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.010856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.012045 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.012046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.013568 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.013572 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.014268 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.014269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.014530 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.015426 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:26:54.015438 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:26:54.017863 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:26:54.020095 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:54.020105 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:54.020108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:54.020443 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:54.020711 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:54.020712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:54.020712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:54.020893 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:54.021217 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:26:54.021225 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:26:54.021409 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 05:26:54.022246 (+ 837us) client_negotiation.cc:240] Negotiation successful
0812 05:26:54.022475 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":222,"threads_started":1}
I20260812 05:26:54.023783 4690 heartbeater.cc:347] Connected to a master server at 127.24.66.126:36219
I20260812 05:26:54.023911 4690 heartbeater.cc:464] Registering TS with master...
I20260812 05:26:54.024134 4690 heartbeater.cc:511] Master 127.24.66.126:36219 requested a full tablet report, sending...
I20260812 05:26:54.024593 4314 ts_manager.cc:194] Registered new tserver with Master: dbf47f36b85e4e4e9b7c6ff8d9aa3878 (127.24.66.66:41659)
I20260812 05:26:54.025097 4314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:37839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:26:54.046159 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:54233/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:36219
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:26:54.267913 4695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:26:54.268435 4695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:26:54.268544 4695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:26:54.271752 4695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:26:54.271999 4695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:26:54.272329 4695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:26:54.275463 4695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.66.84:54233/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:36219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:26:54.276005 4695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:26:54.276396 4695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:26:54.287969 4703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:26:54.287995 4702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:26:54.288552 4695 server_base.cc:1034] running on GCE node
W20260812 05:26:54.290567 4705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:26:54.291092 4695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:26:54.291595 4695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:26:54.292881 4695 hybrid_clock.cc:648] HybridClock initialized: now 1786512414292855 us; error 49 us; skew 500 ppm
Aug 12 05:26:54 dist-test-slave-zpfg krb5kdc[3823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512414, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:26:54.297848 4695 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:26:54.298589 4695 webserver.cc:460] Webserver started at http://127.24.66.67:36905/ using document root <none> and password file <none>
I20260812 05:26:54.299062 4695 fs_manager.cc:359] Metadata directory not provided
I20260812 05:26:54.299368 4695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:26:54.299588 4695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:26:54 dist-test-slave-zpfg krb5kdc[3823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512414, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:26:54.327960 4695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9"
format_stamp: "Formatted at 2026-08-12 05:26:54 on dist-test-slave-zpfg"
server_key: "726a6a139c2f8b1d3915ccbfdb3a0ee1"
server_key_iv: "1e6e8d49d7ebe1c5e6075fe7caff5a72"
server_key_version: "kuduclusterkey@0"
I20260812 05:26:54.331017 4695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9"
format_stamp: "Formatted at 2026-08-12 05:26:54 on dist-test-slave-zpfg"
server_key: "726a6a139c2f8b1d3915ccbfdb3a0ee1"
server_key_iv: "1e6e8d49d7ebe1c5e6075fe7caff5a72"
server_key_version: "kuduclusterkey@0"
I20260812 05:26:54.340790 4695 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.001s
I20260812 05:26:54.359336 4712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:54.359602 4695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 05:26:54.359707 4695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9"
format_stamp: "Formatted at 2026-08-12 05:26:54 on dist-test-slave-zpfg"
server_key: "726a6a139c2f8b1d3915ccbfdb3a0ee1"
server_key_iv: "1e6e8d49d7ebe1c5e6075fe7caff5a72"
server_key_version: "kuduclusterkey@0"
I20260812 05:26:54.359822 4695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:26:54.426604 4695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:26:54.430166 4695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:26:54.430461 4695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:26:54.430939 4695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:26:54.431521 4695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:26:54.431600 4695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:54.431638 4695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:26:54.431663 4695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:26:54.448436 4695 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:41779
I20260812 05:26:54.449097 4831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:41779 every 8 connection(s)
I20260812 05:26:54.449156 4695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:26:54.455479 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 4695
I20260812 05:26:54.455631 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:26:54.455924 24841 external_mini_cluster.cc:1428] Setting key 58404039b605a137133fe695f11024cb
Aug 12 05:26:54 dist-test-slave-zpfg krb5kdc[3823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512414, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:26:54.462906 4550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.450300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:57701 (local address 127.24.66.126:36219)
0812 05:26:54.450409 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:54.450412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:54.450701 (+ 289us) server_negotiation.cc:407] Connection header received
0812 05:26:54.451339 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:54.451340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:54.451363 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:54.451479 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:26:54.452431 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.453381 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.454770 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.455146 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.455257 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.459116 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:26:54.459117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:26:54.459118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:26:54.459126 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:26:54.460920 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:54.461615 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:54.461616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:54.461617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:54.461688 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:54.462073 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:54.462073 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:54.462074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:54.462195 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:26:54.462250 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:54.462741 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 05:26:54.462833 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 05:26:54.463292 4834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.450250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:36219 (local address 127.24.66.67:57701)
0812 05:26:54.450528 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:54.450572 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:54.451221 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:54.451576 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:54.451581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:54.451984 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:26:54.452318 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.452323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.453513 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.453514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.454668 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.454670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.455289 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.455290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.455580 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.456420 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:26:54.456432 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:26:54.458997 (+ 2565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:26:54.461040 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:54.461058 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:54.461060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:54.461500 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:54.461773 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:26:54.461774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:26:54.461775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:26:54.461983 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:26:54.462262 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:26:54.462269 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:26:54.462544 (+ 275us) client_negotiation.cc:769] Sending connection context
0812 05:26:54.462709 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 05:26:54.462970 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":120,"threads_started":1}
I20260812 05:26:54.463941 4832 heartbeater.cc:347] Connected to a master server at 127.24.66.126:36219
I20260812 05:26:54.464093 4832 heartbeater.cc:464] Registering TS with master...
I20260812 05:26:54.464324 4832 heartbeater.cc:511] Master 127.24.66.126:36219 requested a full tablet report, sending...
I20260812 05:26:54.465159 4314 ts_manager.cc:194] Registered new tserver with Master: 49d737cc5fe04ca7a8bd804f6c2f35c9 (127.24.66.67:41779)
I20260812 05:26:54.465576 4314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:57701
I20260812 05:26:54.468649 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:26:54.478390 4550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.470412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.24.66.126:36219)
0812 05:26:54.470459 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:54.470461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:54.470646 (+ 185us) server_negotiation.cc:407] Connection header received
0812 05:26:54.470762 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:54.470763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:54.470773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:54.470826 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:26:54.471634 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.472189 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.472964 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.473211 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.473292 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.474985 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:26:54.474987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:26:54.474988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:26:54.474996 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:26:54.477002 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:54.477456 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:54.477457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:54.477457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:54.477524 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:26:54.477758 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:26:54.477759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:26:54.477759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:26:54.477889 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:26:54.478008 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:54.478161 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 05:26:54.478252 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:26:54.480937 4314 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:26:54.483174 4314 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:26:54.494066 4849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.487042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:41779 (local address 127.0.0.1:36556)
0812 05:26:54.488208 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:54.488255 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:54.488360 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:54.488968 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:54.488969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:54.488972 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:26:54.489078 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.489081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.490519 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.490520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.492002 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.492005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.493696 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.493697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.493793 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.493809 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 05:26:54.493890 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 05:26:54.493962 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1145,"mutex_wait_us":5,"thread_start_us":249,"threads_started":1}
I20260812 05:26:54.494835 4846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.487457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36556 (local address 127.24.66.67:41779)
0812 05:26:54.488020 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:54.488026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:54.488574 (+ 548us) server_negotiation.cc:407] Connection header received
0812 05:26:54.488669 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:54.488672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:54.488691 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:54.488790 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:26:54.489226 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.490423 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.492429 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.493347 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.493427 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.493609 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:54.494639 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0812 05:26:54.494688 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":200,"threads_started":1}
I20260812 05:26:54.495472 4848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.487067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:41539 (local address 127.0.0.1:49574)
0812 05:26:54.488671 (+ 1604us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:54.488684 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:54.488815 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:54.490735 (+ 1920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:54.490736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:54.490738 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:26:54.490774 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.490775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.492547 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.492549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.493883 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.493886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.495201 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.495203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.495269 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.495274 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:26:54.495332 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 05:26:54.495397 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1498,"mutex_wait_us":3,"thread_start_us":786,"threads_started":1}
I20260812 05:26:54.495707 4845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.487277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.24.66.65:41539)
0812 05:26:54.490477 (+ 3200us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:54.490484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:54.490495 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:26:54.490532 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:54.490534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:54.490562 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:54.490654 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:26:54.490850 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.492392 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.493984 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.494999 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.495074 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.495159 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:54.495583 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 05:26:54.495633 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3156,"thread_start_us":73,"threads_started":1}
I20260812 05:26:54.495962 4750 tablet_service.cc:1467] Processing CreateTablet for tablet 43c2d85b592446b4bc1fab015cda2a67 (DEFAULT_TABLE table=test-table [id=3c7789c47dad440cb62bc19b02fa6b61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:26:54.496572 4750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43c2d85b592446b4bc1fab015cda2a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:26:54.499444 4476 tablet_service.cc:1467] Processing CreateTablet for tablet 43c2d85b592446b4bc1fab015cda2a67 (DEFAULT_TABLE table=test-table [id=3c7789c47dad440cb62bc19b02fa6b61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:26:54.499910 4476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43c2d85b592446b4bc1fab015cda2a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:26:54.500726 4844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.487042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:41659 (local address 127.0.0.1:35944)
0812 05:26:54.488207 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:54.488250 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:54.488353 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:54.488856 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:54.488860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:54.488862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:26:54.489219 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.489222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.491375 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.491377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.493160 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.493163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.500380 (+ 7217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.500381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.500498 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.500504 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:26:54.500542 (+ 38us) client_negotiation.cc:240] Negotiation successful
0812 05:26:54.500595 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"mutex_wait_us":2,"thread_start_us":721,"threads_started":1}
I20260812 05:26:54.501358 4847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.487483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35944 (local address 127.24.66.66:41659)
0812 05:26:54.487949 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:54.487955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:54.488337 (+ 382us) server_negotiation.cc:407] Connection header received
0812 05:26:54.488567 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:54.488570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:54.488591 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:54.488677 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:26:54.489345 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.491223 (+ 1878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.493517 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.500187 (+ 6670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.500895 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.501081 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:54.501127 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 05:26:54.501228 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":158,"threads_started":1}
I20260812 05:26:54.502312 4614 tablet_service.cc:1467] Processing CreateTablet for tablet 43c2d85b592446b4bc1fab015cda2a67 (DEFAULT_TABLE table=test-table [id=3c7789c47dad440cb62bc19b02fa6b61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:26:54.503129 4614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43c2d85b592446b4bc1fab015cda2a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:26:54.504825 4850 tablet_bootstrap.cc:492] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9: Bootstrap starting.
I20260812 05:26:54.506268 4850 tablet_bootstrap.cc:654] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9: Neither blocks nor log segments found. Creating new log.
I20260812 05:26:54.506839 4850 log.cc:824] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9: Log is configured to *not* fsync() on all Append() calls
I20260812 05:26:54.508275 4852 tablet_bootstrap.cc:492] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878: Bootstrap starting.
I20260812 05:26:54.509979 4852 tablet_bootstrap.cc:654] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878: Neither blocks nor log segments found. Creating new log.
I20260812 05:26:54.510972 4852 log.cc:824] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878: Log is configured to *not* fsync() on all Append() calls
I20260812 05:26:54.512446 4852 tablet_bootstrap.cc:492] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878: No bootstrap required, opened a new log
I20260812 05:26:54.512554 4852 ts_tablet_manager.cc:1397] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 05:26:54.516474 4852 raft_consensus.cc:348] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41539 } } peers { permanent_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41659 } } peers { permanent_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41779 } }
I20260812 05:26:54.516688 4852 raft_consensus.cc:374] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:26:54.516731 4852 raft_consensus.cc:729] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf47f36b85e4e4e9b7c6ff8d9aa3878, State: Initialized, Role: FOLLOWER
I20260812 05:26:54.516911 4852 consensus_queue.cc:260] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41539 } } peers { permanent_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41659 } } peers { permanent_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41779 } }
I20260812 05:26:54.517272 4852 ts_tablet_manager.cc:1428] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260812 05:26:54.517405 4690 heartbeater.cc:503] Master 127.24.66.126:36219 was elected leader, sending a full tablet report...
I20260812 05:26:54.522574 4854 tablet_bootstrap.cc:492] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c: Bootstrap starting.
I20260812 05:26:54.524338 4854 tablet_bootstrap.cc:654] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c: Neither blocks nor log segments found. Creating new log.
I20260812 05:26:54.525635 4854 log.cc:824] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:26:54.526881 4850 tablet_bootstrap.cc:492] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9: No bootstrap required, opened a new log
I20260812 05:26:54.526995 4850 ts_tablet_manager.cc:1397] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20260812 05:26:54.530267 4854 tablet_bootstrap.cc:492] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c: No bootstrap required, opened a new log
I20260812 05:26:54.530404 4854 ts_tablet_manager.cc:1397] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260812 05:26:54.530776 4850 raft_consensus.cc:348] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41539 } } peers { permanent_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41659 } } peers { permanent_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41779 } }
I20260812 05:26:54.530992 4850 raft_consensus.cc:374] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:26:54.531538 4850 raft_consensus.cc:729] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d737cc5fe04ca7a8bd804f6c2f35c9, State: Initialized, Role: FOLLOWER
I20260812 05:26:54.531741 4850 consensus_queue.cc:260] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41539 } } peers { permanent_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41659 } } peers { permanent_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41779 } }
I20260812 05:26:54.532161 4850 ts_tablet_manager.cc:1428] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260812 05:26:54.532354 4832 heartbeater.cc:503] Master 127.24.66.126:36219 was elected leader, sending a full tablet report...
I20260812 05:26:54.535833 4854 raft_consensus.cc:348] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41539 } } peers { permanent_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41659 } } peers { permanent_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41779 } }
I20260812 05:26:54.536073 4854 raft_consensus.cc:374] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:26:54.536139 4854 raft_consensus.cc:729] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90a8005e4fc4aef8471f4f1e80b860c, State: Initialized, Role: FOLLOWER
I20260812 05:26:54.536309 4854 consensus_queue.cc:260] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41539 } } peers { permanent_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41659 } } peers { permanent_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41779 } }
I20260812 05:26:54.536720 4854 ts_tablet_manager.cc:1428] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260812 05:26:54.536871 4547 heartbeater.cc:503] Master 127.24.66.126:36219 was elected leader, sending a full tablet report...
I20260812 05:26:54.597726 4858 raft_consensus.cc:482] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:26:54.598052 4858 raft_consensus.cc:504] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41539 } } peers { permanent_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41659 } } peers { permanent_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41779 } }
I20260812 05:26:54.598629 4858 leader_election.cc:288] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbf47f36b85e4e4e9b7c6ff8d9aa3878 (127.24.66.66:41659), 49d737cc5fe04ca7a8bd804f6c2f35c9 (127.24.66.67:41779)
I20260812 05:26:54.603794 4847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.599416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:58347 (local address 127.24.66.66:41659)
0812 05:26:54.599468 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:54.599470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:54.599797 (+ 327us) server_negotiation.cc:407] Connection header received
0812 05:26:54.599897 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:54.599898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:54.599908 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:54.599938 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:26:54.600202 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.601111 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.602782 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.603367 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.603419 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.603428 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:54.603677 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 05:26:54.603718 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:26:54.604080 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.599311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:41659 (local address 127.24.66.65:58347)
0812 05:26:54.599739 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:54.599747 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:54.599822 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:54.600009 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:54.600010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:54.600012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:26:54.600109 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.600111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.601281 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.601282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.602676 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.602679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.603522 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.603523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.603598 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.603602 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:26:54.603947 (+ 345us) client_negotiation.cc:240] Negotiation successful
0812 05:26:54.603993 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":130,"threads_started":1}
I20260812 05:26:54.604386 4643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c2d85b592446b4bc1fab015cda2a67" candidate_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" is_pre_election: true
I20260812 05:26:54.604558 4643 raft_consensus.cc:2454] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90a8005e4fc4aef8471f4f1e80b860c in term 0.
I20260812 05:26:54.604802 4426 leader_election.cc:302] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbf47f36b85e4e4e9b7c6ff8d9aa3878, e90a8005e4fc4aef8471f4f1e80b860c; no voters:
I20260812 05:26:54.604979 4858 raft_consensus.cc:2781] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:26:54.605026 4858 raft_consensus.cc:482] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:26:54.605048 4858 raft_consensus.cc:3037] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:26:54.606061 4846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.599063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:48003 (local address 127.24.66.67:41779)
0812 05:26:54.599169 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:54.599172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:54.599670 (+ 498us) server_negotiation.cc:407] Connection header received
0812 05:26:54.599714 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:54.599715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:54.599727 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:54.599789 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:26:54.600393 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.601291 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.604880 (+ 3589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.605533 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.605587 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.605597 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:54.605939 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 05:26:54.605988 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 05:26:54.606418 4858 raft_consensus.cc:504] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41539 } } peers { permanent_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41659 } } peers { permanent_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41779 } }
I20260812 05:26:54.606503 4859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.598820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:41779 (local address 127.24.66.65:48003)
0812 05:26:54.599533 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:54.599544 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:26:54.599609 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:26:54.599856 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:26:54.599858 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:26:54.599860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:26:54.600242 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.600246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.601496 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.601497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.602826 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:26:54.602829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.605679 (+ 2850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.605680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:26:54.605831 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.605836 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:26:54.606208 (+ 372us) client_negotiation.cc:240] Negotiation successful
0812 05:26:54.606258 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":174,"threads_started":1}
I20260812 05:26:54.606724 4643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c2d85b592446b4bc1fab015cda2a67" candidate_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878"
I20260812 05:26:54.606820 4643 raft_consensus.cc:3037] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:26:54.606995 4784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c2d85b592446b4bc1fab015cda2a67" candidate_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" is_pre_election: true
I20260812 05:26:54.607244 4784 raft_consensus.cc:2454] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90a8005e4fc4aef8471f4f1e80b860c in term 0.
I20260812 05:26:54.607828 4643 raft_consensus.cc:2454] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90a8005e4fc4aef8471f4f1e80b860c in term 1.
I20260812 05:26:54.608021 4426 leader_election.cc:302] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dbf47f36b85e4e4e9b7c6ff8d9aa3878, e90a8005e4fc4aef8471f4f1e80b860c; no voters:
I20260812 05:26:54.608367 4784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c2d85b592446b4bc1fab015cda2a67" candidate_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9"
I20260812 05:26:54.608450 4784 raft_consensus.cc:3037] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:26:54.609637 4784 raft_consensus.cc:2454] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90a8005e4fc4aef8471f4f1e80b860c in term 1.
I20260812 05:26:54.609839 4858 leader_election.cc:288] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [CANDIDATE]: Term 1 election: Requested vote from peers dbf47f36b85e4e4e9b7c6ff8d9aa3878 (127.24.66.66:41659), 49d737cc5fe04ca7a8bd804f6c2f35c9 (127.24.66.67:41779)
I20260812 05:26:54.609912 4858 raft_consensus.cc:2781] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:26:54.609954 4858 raft_consensus.cc:686] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [term 1 LEADER]: Becoming Leader. State: Replica: e90a8005e4fc4aef8471f4f1e80b860c, State: Running, Role: LEADER
I20260812 05:26:54.610044 4858 consensus_queue.cc:237] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41539 } } peers { permanent_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41659 } } peers { permanent_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41779 } }
I20260812 05:26:54.610965 4314 catalog_manager.cc:5495] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c reported cstate change: term changed from 0 to 1, leader changed from <none> to e90a8005e4fc4aef8471f4f1e80b860c (127.24.66.65). New cstate: current_term: 1 leader_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90a8005e4fc4aef8471f4f1e80b860c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41779 } health_report { overall_health: UNKNOWN } } }
I20260812 05:26:54.625160 4845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.621341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.24.66.65:41539)
0812 05:26:54.621388 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:26:54.621390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:26:54.621401 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:26:54.621515 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:26:54.621516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:26:54.621539 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:26:54.621619 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:26:54.621799 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.622700 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.623971 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:26:54.624389 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:26:54.624457 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:26:54.624697 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:26:54.624850 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:26:54.624965 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 05:26:54.625016 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 05:26:54.625042 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:26:54.627293 4784 raft_consensus.cc:1261] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9 [term 1 FOLLOWER]: Refusing update from remote peer e90a8005e4fc4aef8471f4f1e80b860c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:26:54.627357 4643 raft_consensus.cc:1261] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878 [term 1 FOLLOWER]: Refusing update from remote peer e90a8005e4fc4aef8471f4f1e80b860c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:26:54.627715 4858 consensus_queue.cc:1035] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbf47f36b85e4e4e9b7c6ff8d9aa3878" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:26:54.627888 4858 consensus_queue.cc:1035] T 43c2d85b592446b4bc1fab015cda2a67 P e90a8005e4fc4aef8471f4f1e80b860c [LEADER]: Connected to new peer: Peer: permanent_uuid: "49d737cc5fe04ca7a8bd804f6c2f35c9" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:26:54.631223 4864 mvcc.cc:205] Tried to move back new op lower bound from 7317554850310344704 to 7317554850243461120. Current Snapshot: MvccSnapshot[applied={T|T < 7317554850310344704}]
I20260812 05:26:54.632352 4865 mvcc.cc:205] Tried to move back new op lower bound from 7317554850310344704 to 7317554850243461120. Current Snapshot: MvccSnapshot[applied={T|T < 7317554850310344704}]
I20260812 05:26:54.634128 4866 mvcc.cc:205] Tried to move back new op lower bound from 7317554850310344704 to 7317554850243461120. Current Snapshot: MvccSnapshot[applied={T|T < 7317554850310344704}]
I20260812 05:26:54.638036 4321 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:26:54.638147 4321 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:26:54.639005 4321 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b92e6d12a9e4a61aeb99851553b1ec6: Sending DeleteTablet for 3 replicas of tablet 43c2d85b592446b4bc1fab015cda2a67
I20260812 05:26:54.639460 4750 tablet_service.cc:1514] Processing DeleteTablet for tablet 43c2d85b592446b4bc1fab015cda2a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:26:54 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:36556
I20260812 05:26:54.639532 4476 tablet_service.cc:1514] Processing DeleteTablet for tablet 43c2d85b592446b4bc1fab015cda2a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:26:54 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:49574
I20260812 05:26:54.639843 4614 tablet_service.cc:1514] Processing DeleteTablet for tablet 43c2d85b592446b4bc1fab015cda2a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:26:54 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:35944
I20260812 05:26:54.640188 4874 tablet_replica.cc:330] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9: stopping tablet replica
I20260812 05:26:54.640216 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 4408
I20260812 05:26:54.640372 4874 raft_consensus.cc:2229] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:26:54.640523 4874 raft_consensus.cc:2258] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:26:54.642549 4876 tablet_replica.cc:330] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878: stopping tablet replica
I20260812 05:26:54.642946 4874 ts_tablet_manager.cc:1905] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:26:54.645625 4874 ts_tablet_manager.cc:1918] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:26:54.645788 4874 log.cc:1196] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/43c2d85b592446b4bc1fab015cda2a67
I20260812 05:26:54.646863 4874 ts_tablet_manager.cc:1939] T 43c2d85b592446b4bc1fab015cda2a67 P 49d737cc5fe04ca7a8bd804f6c2f35c9: Deleting consensus metadata
I20260812 05:26:54.647481 4308 catalog_manager.cc:4841] TS 49d737cc5fe04ca7a8bd804f6c2f35c9 (127.24.66.67:41779): tablet 43c2d85b592446b4bc1fab015cda2a67 (table test-table [id=3c7789c47dad440cb62bc19b02fa6b61]) successfully deleted
I20260812 05:26:54.649647 4876 raft_consensus.cc:2229] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:26:54.649818 4876 raft_consensus.cc:2258] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 05:26:54.650135 4305 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:41539 (error 108)
I20260812 05:26:54.650543 4876 ts_tablet_manager.cc:1905] T 43c2d85b592446b4bc1fab015cda2a67 P dbf47f36b85e4e4e9b7c6ff8d9aa3878: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:26:54.650976 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 4553
I20260812 05:26:54.658021 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 4695
I20260812 05:26:54.660324 4844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.660022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:41659 (local address 127.0.0.1:35950)
0812 05:26:54.660088 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:54.660273 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.66:41659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 05:26:54.660621 4304 catalog_manager.cc:4584] TS dbf47f36b85e4e4e9b7c6ff8d9aa3878 (127.24.66.66:41659): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 43c2d85b592446b4bc1fab015cda2a67: Network error: Client connection negotiation failed: client connection to 127.24.66.66:41659: connect: Connection refused (error 111)
I20260812 05:26:54.661512 4844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:26:54.661370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:41539 (local address 127.0.0.1:49592)
0812 05:26:54.661445 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 05:26:54.661483 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:41539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 05:26:54.666190 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 4288
I20260812 05:26:54.672580 24841 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 05:26:55.017830 24841 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 05:26:55.017953 24841 mini_ranger.cc:66] Stopping Ranger...
I20260812 05:26:55.364575 24841 mini_ranger.cc:68] Stopped Ranger
2026-08-12 05:26:55.364 UTC [3854] LOG: received smart shutdown request
2026-08-12 05:26:55.369 UTC [3854] LOG: background worker "logical replication launcher" (PID 3863) exited with exit code 1
2026-08-12 05:26:55.369 UTC [3858] LOG: shutting down
2026-08-12 05:26:55.546 UTC [3854] LOG: database system is shut down
2026-08-12T05:26:55Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53506 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:26:55 dist-test-slave-zpfg krb5kdc[4886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:26:55 dist-test-slave-zpfg krb5kdc[4886](info): set up 2 sockets
Aug 12 05:26:55 dist-test-slave-zpfg krb5kdc[4886](info): commencing operation
krb5kdc: starting...
W20260812 05:26:56.850901 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:26:56 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:26:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:26:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.66.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.66.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/rangeradmin_127.24.66.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/rangeradmin_127.24.66.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.66.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.66.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/rangerlookup_127.24.66.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/rangerlookup_127.24.66.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.66.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.66.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/HTTP_127.24.66.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/HTTP_127.24.66.84@KRBTEST.COM.keytab.
I20260812 05:26:57.042411 24841 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task_bqoao/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/postgres -l logfile start
2026-08-12 05:26:59.963 UTC [4917] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 05:26:59.963 UTC [4917] LOG: listening on IPv4 address "127.24.66.84", port 60913
2026-08-12 05:26:59.968 UTC [4917] LOG: listening on Unix socket "/tmp/.s.PGSQL.60913"
2026-08-12 05:26:59.991 UTC [4920] LOG: database system was shut down at 2026-08-12 05:26:58 UTC
2026-08-12 05:26:59.996 UTC [4917] LOG: database system is ready to accept connections
I20260812 05:27:00.985873 24841 mini_postgres.cc:96] Postgres bound to 60913
2026-08-12 05:27:00.993 UTC [4928] FATAL: database "slave" does not exist
127.24.66.84:60913 - accepting connections
I20260812 05:27:00.993639 24841 mini_ranger.cc:161] Starting Ranger...
I20260812 05:27:01.010694 24841 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 05:27:01.590443 24841 mini_ranger.cc:87] Created ranger Postgres database
I20260812 05:27:01.590547 24841 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-admin
2026-08-12 05:27:01,886 [I] DB FLAVOR :POSTGRES
2026-08-12 05:27:01,887 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 05:27:01,887 [I] Checking connection..
2026-08-12 05:27:01,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 05:27:02,167 [I] Checking connection passed.
2026-08-12 05:27:02,167 [I] --------- Verifying version history table ---------
2026-08-12 05:27:02,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:27:02,425 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 05:27:02,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:27:02,688 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 05:27:02,688 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 05:27:02,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 05:27:02.974 UTC [4994] WARNING: there is no transaction in progress
2026-08-12 05:27:02,981 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 05:27:02,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 05:27:03,229 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 05:27:03,230 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 05:27:03,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 05:27:03,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 05:27:03,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zpfg.c.gcp-upstream.internal','N') ;"
2026-08-12 05:27:04,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 05:27:04,298 [I] Table x_portal_user does not exist in database ranger
2026-08-12 05:27:04,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 05:27:04,561 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 05:27:04,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 05:27:04,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 05:27:05,119 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 05:27:05,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 05:27:06.242 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.260 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.279 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.298 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.317 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.338 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.413 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.422 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.443 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.456 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.469 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.487 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.500 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.508 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.516 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:06.524 UTC [5107] WARNING: there is no transaction in progress
2026-08-12 05:27:07,371 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 05:27:07,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 05:27:07,640 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 05:27:07,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 05:27:07,904 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 05:27:07,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 05:27:08,205 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 05:27:08,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 05:27:08,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 05:27:08,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zpfg.c.gcp-upstream.internal';"
2026-08-12 05:27:08,960 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 05:27:08,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 05:27:09,221 [I] DB_PATCHES have already been applied
I20260812 05:27:09.227890 24841 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-admin:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task_bqoao/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 05:27:09.227962 24841 mini_ranger.cc:193] Using host: 127.24.66.84
I20260812 05:27:09.231904 24841 mini_ranger.cc:227] Ranger admin URL: http://127.24.66.84:48799
Aug 12, 2026 5:27:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:27:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:27:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 5:27:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 5:27:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 5:27:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 5:27:10 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 05:27:10.402038 24841 mini_ranger.cc:160] Time spent starting Ranger: real 9.408s user 0.001s sys 0.011s
Aug 12, 2026 5:27:32 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 5:27:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 12, 2026 5:27:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 5:27:33 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 5:27:34 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 05:27:34.651010 24841 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.66.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.66.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/rangerkms_127.24.66.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.66.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/rangerkms_127.24.66.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.66.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.66.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.66.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/HTTP_127.24.66.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.66.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/HTTP_127.24.66.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 05:27:34 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512454, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 05:27:34.772475 24841 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 05:27:34.786778 24841 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 05:27:35.779970 24841 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 05:27:35.780087 24841 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-kms
I20260812 05:27:35.780112 24841 mini_ranger_kms.cc:227] Using postgres at 127.24.66.84:60913
2026-08-12 05:27:35,995 [I] DB FLAVOR :POSTGRES
2026-08-12 05:27:35,995 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 05:27:35,995 [I] Checking connection
2026-08-12 05:27:35,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 05:27:36,279 [I] connection success
2026-08-12 05:27:36,280 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 05:27:36,280 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 05:27:36,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 05:27:36,537 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 05:27:36,538 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 05:27:36,538 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 05:27:36,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.66.84:60913/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 05:27:36,824 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 05:27:37.379958 24841 mini_ranger_kms.cc:310] Created kms service
I20260812 05:27:37.599522 24841 mini_ranger_kms.cc:326] Created kudu user
I20260812 05:27:37.650415 24841 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 05:27:37 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512454, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:27:38.115262 24841 mini_ranger_kms.cc:384] Added ranger policy
I20260812 05:27:38.115487 24841 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-kms:/tmp/dist-test-task_bqoao/build/release/bin/postgresql.jar:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task_bqoao/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 05:27:38.115509 24841 mini_ranger_kms.cc:240] Using host: 127.24.66.84
I20260812 05:27:38.120422 24841 mini_ranger_kms.cc:276] Ranger KMS PID: 5292
I20260812 05:27:38.120527 24841 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.66.84:36543
Aug 12, 2026 5:27:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 5:27:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 5:27:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 5:27:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 5:27:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 5:27:39 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 05:27:40 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786512460, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.66.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:40.632390 24841 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.860s user 0.009s sys 0.184s
I20260812 05:27:40.632495 24841 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 05:27:40.632540 24841 mini_ranger_kms.cc:401] 127.24.66.84:36543/kms/v1/keys
Aug 12 05:27:40 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512460, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.66.84@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
Aug 12 05:27:41 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512460, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.66.84@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
Aug 12 05:27:43 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:27:43.845366 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:44667
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:44667
--ranger_config_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:36543/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:43.934677 5342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:43.934943 5342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:43.934985 5342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:43.936267 5342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:27:43.936343 5342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:43.936365 5342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:43.936385 5342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:27:43.936414 5342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:27:43.938411 5342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.66.84:36543/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:44667
--ranger_config_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:44667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:43.938738 5342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:43.938990 5342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:43.945818 5348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:43.945871 5350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:43.945920 5342 server_base.cc:1034] running on GCE node
W20260812 05:27:43.946056 5347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:43.946377 5342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:43.946727 5342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:43.947927 5342 hybrid_clock.cc:648] HybridClock initialized: now 1786512463947893 us; error 40 us; skew 500 ppm
Aug 12 05:27:43 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512463, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:43.957239 5342 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:27:43.957906 5342 webserver.cc:460] Webserver started at http://127.24.66.126:34419/ using document root <none> and password file <none>
I20260812 05:27:43.958158 5342 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:43.958201 5342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:43.958343 5342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:27:43 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512463, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:27:44.016366 5342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "d0c360944d8949dfb030db42e66cda30"
format_stamp: "Formatted at 2026-08-12 05:27:44 on dist-test-slave-zpfg"
server_key: "f3fe7544659a1b6410eb37815195d348"
server_key_iv: "ec4395c4502ceb799a014a8d00f879fc"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:44.016970 5342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "d0c360944d8949dfb030db42e66cda30"
format_stamp: "Formatted at 2026-08-12 05:27:44 on dist-test-slave-zpfg"
server_key: "f3fe7544659a1b6410eb37815195d348"
server_key_iv: "ec4395c4502ceb799a014a8d00f879fc"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:44.018952 5342 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 05:27:44.036352 5357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:44.036640 5342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:44.036768 5342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "d0c360944d8949dfb030db42e66cda30"
format_stamp: "Formatted at 2026-08-12 05:27:44 on dist-test-slave-zpfg"
server_key: "f3fe7544659a1b6410eb37815195d348"
server_key_iv: "ec4395c4502ceb799a014a8d00f879fc"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:44.036878 5342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:44.067467 5342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:44.070109 5342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:44.070313 5342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:44.076143 5342 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:44667
I20260812 05:27:44.076166 5412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:44667 every 8 connection(s)
I20260812 05:27:44.076682 5342 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:27:44.077513 5413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:44.080294 5413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30: Bootstrap starting.
I20260812 05:27:44.081013 5413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:44.081333 5413 log.cc:824] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:44.082098 5413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30: No bootstrap required, opened a new log
I20260812 05:27:44.083391 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 5342
I20260812 05:27:44.083511 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:27:44.083699 24841 external_mini_cluster.cc:1428] Setting key d9d45f6e4fb0314e3ac11dab7bbff962
I20260812 05:27:44.084954 5413 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } }
I20260812 05:27:44.085175 5413 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:44.085253 5413 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c360944d8949dfb030db42e66cda30, State: Initialized, Role: FOLLOWER
I20260812 05:27:44.085454 5413 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } }
I20260812 05:27:44.085554 5413 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:27:44.085611 5413 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:27:44.085685 5413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:44.086505 5413 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } }
I20260812 05:27:44.086619 5413 leader_election.cc:302] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0c360944d8949dfb030db42e66cda30; no voters:
I20260812 05:27:44.086831 5413 leader_election.cc:288] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:27:44.087020 5419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:27:44.087093 5413 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:27:44.087230 5419 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 1 LEADER]: Becoming Leader. State: Replica: d0c360944d8949dfb030db42e66cda30, State: Running, Role: LEADER
I20260812 05:27:44.087417 5419 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } }
I20260812 05:27:44.088737 5419 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c360944d8949dfb030db42e66cda30. Latest consensus state: current_term: 1 leader_uuid: "d0c360944d8949dfb030db42e66cda30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } } }
I20260812 05:27:44.088822 5419 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:44.089066 5434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:27:44.089039 5418 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0c360944d8949dfb030db42e66cda30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } } }
I20260812 05:27:44.089118 5418 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:44.089386 5413 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 05:27:44.089643 5434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:27:44.091527 5434 catalog_manager.cc:1347] Generated new cluster ID: f03486133c96462e8dbfa43fdf2fe1ba
I20260812 05:27:44.091578 5434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:27:44 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:44.098326 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:44.084325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60046 (local address 127.24.66.126:44667)
0812 05:27:44.084540 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:44.084553 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:44.084594 (+ 41us) server_negotiation.cc:407] Connection header received
0812 05:27:44.084917 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:44.084926 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:44.085002 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:44.085170 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:44.086133 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:44.087630 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:44.088835 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:44.089555 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:44.089677 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:44.093151 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:44.093156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:44.093156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:44.093170 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:44.096071 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:44.096542 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:44.096546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:44.096546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:44.096697 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:44.096912 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:44.096912 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:44.096913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:44.097244 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:44.097391 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:44.097949 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 05:27:44.098091 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260812 05:27:44.110834 5434 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:27:44.112275 5434 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:27:44.119451 5434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30: Generated new TSK 0
I20260812 05:27:44.119779 5434 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:27:44.998142 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:36543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44667
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:45.132380 5460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:45.132706 5460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:45.132750 5460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:45.134691 5460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:45.134765 5460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:45.134811 5460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:27:45.137550 5460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.66.84:36543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:45.137859 5460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:45.138222 5460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:45.145376 5467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:45.145385 5465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:45.145354 5469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:45.146006 5460 server_base.cc:1034] running on GCE node
I20260812 05:27:45.146256 5460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:45.146606 5460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:45.147797 5460 hybrid_clock.cc:648] HybridClock initialized: now 1786512465147773 us; error 36 us; skew 500 ppm
Aug 12 05:27:45 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512465, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:45.151736 5460 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:27:45.152581 5460 webserver.cc:460] Webserver started at http://127.24.66.65:40527/ using document root <none> and password file <none>
I20260812 05:27:45.153013 5460 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:45.153110 5460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:45.153306 5460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:27:45 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512465, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:27:45.174175 5460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "ff64417c158b45eebaff0af97b8703dd"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "361e97d39ef272fa6760897aaca15f3a"
server_key_iv: "517c12c416e86aebacd81b1a8556d01f"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:45.174772 5460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "ff64417c158b45eebaff0af97b8703dd"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "361e97d39ef272fa6760897aaca15f3a"
server_key_iv: "517c12c416e86aebacd81b1a8556d01f"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:45.179180 5460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260812 05:27:45.193588 5478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:45.193770 5460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:45.193861 5460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "ff64417c158b45eebaff0af97b8703dd"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "361e97d39ef272fa6760897aaca15f3a"
server_key_iv: "517c12c416e86aebacd81b1a8556d01f"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:45.193960 5460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:45.227591 5460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:45.230300 5460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:45.230564 5460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:45.230898 5460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:45.231350 5460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:45.231419 5460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:45.231467 5460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:45.231498 5460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:45.238327 5460 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:46407
I20260812 05:27:45.241070 5598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:46407 every 8 connection(s)
I20260812 05:27:45.242039 5460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:27:45.249084 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 5460
I20260812 05:27:45.249200 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:27:45.249416 24841 external_mini_cluster.cc:1428] Setting key 1c34bdf9b4d858d04d4aa350868b7510
Aug 12 05:27:45 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512465, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:45.255784 5601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:45.243349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:44667 (local address 127.24.66.65:46159)
0812 05:27:45.243842 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:45.243903 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:45.244554 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:45.245010 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:45.245014 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:45.245285 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:45.245579 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:45.245585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.246582 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.246583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:45.247812 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:45.247815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.248377 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.248379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:45.248531 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:45.249526 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:45.249537 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:45.251838 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:45.254007 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:45.254016 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:45.254019 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:45.254409 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:45.254625 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:45.254626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:45.254626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:45.254759 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:45.255051 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:45.255057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:45.255240 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 05:27:45.255354 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 05:27:45.255525 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":95,"threads_started":1}
I20260812 05:27:45.256404 5602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:45.243670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:46159 (local address 127.24.66.126:44667)
0812 05:27:45.244754 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:45.244757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:45.244765 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:27:45.244794 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:45.244794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:45.244808 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:45.244867 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:45.245700 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.246470 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.247922 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.248224 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.248286 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:45.251960 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:45.251962 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:45.251962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:45.251971 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:45.253888 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:45.254493 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:45.254494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:45.254495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:45.254561 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:45.254823 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:45.254824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:45.254824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:45.254959 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:45.255015 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:45.256209 (+ 1194us) server_negotiation.cc:299] Negotiation successful
0812 05:27:45.256303 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1060,"thread_start_us":73,"threads_started":1}
I20260812 05:27:45.257205 5599 heartbeater.cc:347] Connected to a master server at 127.24.66.126:44667
I20260812 05:27:45.257302 5599 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:45.257509 5599 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
I20260812 05:27:45.258167 5374 ts_manager.cc:194] Registered new tserver with Master: ff64417c158b45eebaff0af97b8703dd (127.24.66.65:46407)
I20260812 05:27:45.259027 5374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:46159
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:27:45.313683 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:36543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44667
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:27:45 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786512465, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 05:27:45.457716 5607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:45.458161 5607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:45.458268 5607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:45.460446 5607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:45.460724 5607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:45.460871 5607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:27:45.463876 5607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.66.84:36543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:45.464622 5607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:45.465013 5607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:45.472229 5612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:45.472241 5607 server_base.cc:1034] running on GCE node
W20260812 05:27:45.472110 5615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:45.472101 5613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:45.472817 5607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:45.473134 5607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:45.474318 5607 hybrid_clock.cc:648] HybridClock initialized: now 1786512465474297 us; error 38 us; skew 500 ppm
Aug 12 05:27:45 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512465, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:45.477883 5607 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:27:45.478663 5607 webserver.cc:460] Webserver started at http://127.24.66.66:35745/ using document root <none> and password file <none>
I20260812 05:27:45.478950 5607 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:45.479019 5607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:45.479221 5607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:27:45 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512465, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:27:45.495528 5607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "1164416641e647be8454e51c859b9b7b"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "fcfd760cb569a0b69591b108565f86c0"
server_key_iv: "d1be3ccd347321343a26392bc4302a89"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:45.496258 5607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "1164416641e647be8454e51c859b9b7b"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "fcfd760cb569a0b69591b108565f86c0"
server_key_iv: "d1be3ccd347321343a26392bc4302a89"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:45.498600 5607 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 05:27:45.514228 5623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:45.514536 5607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:45.514662 5607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "1164416641e647be8454e51c859b9b7b"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "fcfd760cb569a0b69591b108565f86c0"
server_key_iv: "d1be3ccd347321343a26392bc4302a89"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:45.514801 5607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:45.533552 5607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:45.536734 5607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:45.536981 5607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:45.537379 5607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:45.537885 5607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:45.538029 5607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:45.538143 5607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:45.538219 5607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:45.546926 5607 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:45437
I20260812 05:27:45.547505 5607 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:27:45.550050 5742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:45437 every 8 connection(s)
I20260812 05:27:45.553316 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 5607
I20260812 05:27:45.553431 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:27:45.553635 24841 external_mini_cluster.cc:1428] Setting key d6d75c269f438a9cbfbb9b227c75acea
Aug 12 05:27:45 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512465, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:45.563397 5602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:45.550799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:43353 (local address 127.24.66.126:44667)
0812 05:27:45.550857 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:45.550860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:45.551154 (+ 294us) server_negotiation.cc:407] Connection header received
0812 05:27:45.551957 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:45.551959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:45.551979 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:45.552020 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:45.553078 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.553986 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.555762 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.556158 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.556209 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:45.559225 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:45.559227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:45.559227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:45.559237 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:45.561336 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:45.561963 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:45.561964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:45.561965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:45.562035 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:45.562410 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:45.562411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:45.562412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:45.562581 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:45.562632 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:45.563112 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 05:27:45.563230 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 05:27:45.564419 5746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:45.550756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:44667 (local address 127.24.66.66:43353)
0812 05:27:45.550988 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:45.551047 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:45.551804 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:45.552245 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:45.552249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:45.552544 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:45.552880 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:45.552886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.554176 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.554177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:45.555642 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:45.555645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.556325 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.556326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:45.556492 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:45.557237 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:45.557250 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:45.559103 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:45.561449 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:45.561458 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:45.561461 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:45.561849 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:45.562124 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:45.562125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:45.562125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:45.562285 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:45.562700 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:45.562708 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:45.562912 (+ 204us) client_negotiation.cc:769] Sending connection context
0812 05:27:45.563869 (+ 957us) client_negotiation.cc:240] Negotiation successful
0812 05:27:45.564093 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
I20260812 05:27:45.565239 5744 heartbeater.cc:347] Connected to a master server at 127.24.66.126:44667
I20260812 05:27:45.565357 5744 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:45.565565 5744 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
I20260812 05:27:45.566035 5374 ts_manager.cc:194] Registered new tserver with Master: 1164416641e647be8454e51c859b9b7b (127.24.66.66:45437)
I20260812 05:27:45.566670 5374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:43353
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:27:45.594770 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:36543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44667
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:45.732576 5750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:45.732978 5750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:45.733076 5750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:45.735140 5750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:45.735424 5750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:45.735535 5750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:27:45.737991 5750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.66.84:36543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:45.738302 5750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:45.738605 5750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:45.744374 5757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:45.744374 5759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:45.746965 5756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:45.747900 5750 server_base.cc:1034] running on GCE node
I20260812 05:27:45.748157 5750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:45.748486 5750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:45.751788 5750 hybrid_clock.cc:648] HybridClock initialized: now 1786512465751764 us; error 39 us; skew 500 ppm
Aug 12 05:27:45 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512465, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:45.754984 5750 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:27:45.755601 5750 webserver.cc:460] Webserver started at http://127.24.66.67:41979/ using document root <none> and password file <none>
I20260812 05:27:45.755908 5750 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:45.755995 5750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:45.756162 5750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 05:27:45 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512465, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:27:45.771454 5750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "39d7bc7d30f4441a95b70fb4d278f465"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "3c0ae0e860cf43fbe774ae427eb27e9b"
server_key_iv: "21420f97c156d86390f241db40229b95"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:45.771952 5750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "39d7bc7d30f4441a95b70fb4d278f465"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "3c0ae0e860cf43fbe774ae427eb27e9b"
server_key_iv: "21420f97c156d86390f241db40229b95"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:45.799043 5750 fs_manager.cc:691] Time spent creating directory manager: real 0.027s user 0.002s sys 0.000s
I20260812 05:27:45.814180 5766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:45.815127 5750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:27:45.815263 5750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "39d7bc7d30f4441a95b70fb4d278f465"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "3c0ae0e860cf43fbe774ae427eb27e9b"
server_key_iv: "21420f97c156d86390f241db40229b95"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:45.815371 5750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:45.838835 5750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:45.841665 5750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:45.841972 5750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:45.842293 5750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:45.842841 5750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:45.842939 5750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:45.843011 5750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:45.843072 5750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:45.850275 5750 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:41623
I20260812 05:27:45.851083 5750 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:27:45.853621 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 5750
I20260812 05:27:45.853767 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:27:45.853989 24841 external_mini_cluster.cc:1428] Setting key 1620cac24ae569d1cd5e8468549854b1
I20260812 05:27:45.854140 5886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:41623 every 8 connection(s)
Aug 12 05:27:45 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512465, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:45.867177 5602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:45.855471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:52379 (local address 127.24.66.126:44667)
0812 05:27:45.855522 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:45.855524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:45.855989 (+ 465us) server_negotiation.cc:407] Connection header received
0812 05:27:45.856772 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:45.856773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:45.856786 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:45.856817 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:45.857709 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.858546 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.860081 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.860407 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.860457 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:45.863464 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:45.863465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:45.863466 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:45.863475 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:45.865321 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:45.865910 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:45.865912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:45.865912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:45.865979 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:45.866306 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:45.866307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:45.866307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:45.866489 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:45.866549 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:45.866984 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 05:27:45.867076 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:27:45.868227 5889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:45.855483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:44667 (local address 127.24.66.67:52379)
0812 05:27:45.855863 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:45.855906 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:45.856640 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:45.857033 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:45.857038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:45.857307 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:45.857593 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:45.857599 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.858665 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.858666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:45.859968 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:45.859971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:45.860586 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:45.860587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:45.860746 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:45.861458 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:45.861469 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:45.863321 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:45.865468 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:45.865476 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:45.865479 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:45.865823 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:45.866056 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:45.866057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:45.866057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:45.866203 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:45.866605 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:45.866612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:45.866798 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 05:27:45.867711 (+ 913us) client_negotiation.cc:240] Negotiation successful
0812 05:27:45.867946 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":179,"threads_started":1}
I20260812 05:27:45.868974 5887 heartbeater.cc:347] Connected to a master server at 127.24.66.126:44667
I20260812 05:27:45.869083 5887 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:45.869321 5887 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
I20260812 05:27:45.869796 5374 ts_manager.cc:194] Registered new tserver with Master: 39d7bc7d30f4441a95b70fb4d278f465 (127.24.66.67:41623)
I20260812 05:27:45.870312 5374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:52379
I20260812 05:27:45.876992 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:27:45.877790 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 5460
I20260812 05:27:45.898226 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 5607
I20260812 05:27:45.909932 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 5750
I20260812 05:27:45.920176 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 5342
I20260812 05:27:45.929981 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:44667
--webserver_interface=127.24.66.126
--webserver_port=34419
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:44667
--ranger_config_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:36543/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.66.84:36543/kms with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:46.017575 5890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:46.017818 5890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:46.017843 5890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:46.019209 5890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:27:46.019287 5890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:46.019313 5890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:46.019335 5890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:27:46.019362 5890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:27:46.021191 5890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.66.84:36543/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:44667
--ranger_config_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:44667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=34419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:46.021492 5890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:46.021739 5890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:46.027419 5897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:46.027413 5899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:46.027491 5890 server_base.cc:1034] running on GCE node
W20260812 05:27:46.027413 5901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:46.027943 5890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:46.028330 5890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:46.029518 5890 hybrid_clock.cc:648] HybridClock initialized: now 1786512466029495 us; error 59 us; skew 500 ppm
Aug 12 05:27:46 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:46.032508 5890 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:27:46.033001 5890 webserver.cc:460] Webserver started at http://127.24.66.126:34419/ using document root <none> and password file <none>
I20260812 05:27:46.033239 5890 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:46.033324 5890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 05:27:46 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:27:46.083438 5890 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:46.084251 5909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:46.084553 5890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:46.084692 5890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "d0c360944d8949dfb030db42e66cda30"
format_stamp: "Formatted at 2026-08-12 05:27:44 on dist-test-slave-zpfg"
server_key: "f3fe7544659a1b6410eb37815195d348"
server_key_iv: "ec4395c4502ceb799a014a8d00f879fc"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:46.085194 5890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:46.111824 5890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:46.114058 5890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:46.114200 5890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:46.119287 5890 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:44667
I20260812 05:27:46.119323 5964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:44667 every 8 connection(s)
I20260812 05:27:46.119812 5890 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:27:46.121186 5965 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:27:46.121797 5965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30: Bootstrap starting.
I20260812 05:27:46.128306 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 5890
I20260812 05:27:46.128660 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:36543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:46407
--local_ip_for_outbound_sockets=127.24.66.65
--tserver_master_addrs=127.24.66.126:44667
--webserver_port=40527
--webserver_interface=127.24.66.65
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.66.84:36543/kms with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:27:46.130616 5965 log.cc:824] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:46.132454 5965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:27:46.132774 5965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30: Bootstrap complete.
I20260812 05:27:46.135416 5965 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } }
I20260812 05:27:46.135700 5965 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c360944d8949dfb030db42e66cda30, State: Initialized, Role: FOLLOWER
I20260812 05:27:46.135922 5965 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } }
I20260812 05:27:46.136022 5965 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:27:46.136080 5965 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:27:46.136149 5965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:27:46.137212 5965 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } }
I20260812 05:27:46.137339 5965 leader_election.cc:302] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0c360944d8949dfb030db42e66cda30; no voters:
I20260812 05:27:46.137565 5965 leader_election.cc:288] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 05:27:46.137606 5969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 05:27:46.137737 5969 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [term 2 LEADER]: Becoming Leader. State: Replica: d0c360944d8949dfb030db42e66cda30, State: Running, Role: LEADER
I20260812 05:27:46.137877 5965 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:27:46.137914 5969 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } }
I20260812 05:27:46.138293 5970 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d0c360944d8949dfb030db42e66cda30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } } }
I20260812 05:27:46.138314 5971 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c360944d8949dfb030db42e66cda30. Latest consensus state: current_term: 2 leader_uuid: "d0c360944d8949dfb030db42e66cda30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c360944d8949dfb030db42e66cda30" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44667 } } }
I20260812 05:27:46.138404 5970 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:46.138444 5971 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30 [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:46.139106 5976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:27:46.139765 5965 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 05:27:46.139807 5976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:27:46.140130 5976 catalog_manager.cc:1259] Loaded cluster ID: f03486133c96462e8dbfa43fdf2fe1ba
I20260812 05:27:46.140187 5976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:27:46.141623 5976 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:27:46.141834 5976 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30: Loaded TSK: 0
I20260812 05:27:46.142181 5976 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 05:27:46.255468 5967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:46.255748 5967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:46.255795 5967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:46.257179 5967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:46.257256 5967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:46.257303 5967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:27:46.259745 5967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.66.84:36543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:46407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=40527
--tserver_master_addrs=127.24.66.126:44667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:46.260200 5967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:46.260535 5967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:46.268268 6006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:46.268416 5967 server_base.cc:1034] running on GCE node
W20260812 05:27:46.268280 6003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:46.268637 6004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:46.269176 5967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:46.269487 5967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:46.270658 5967 hybrid_clock.cc:648] HybridClock initialized: now 1786512466270633 us; error 38 us; skew 500 ppm
Aug 12 05:27:46 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:46.274171 5967 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:27:46.274801 5967 webserver.cc:460] Webserver started at http://127.24.66.65:40527/ using document root <none> and password file <none>
I20260812 05:27:46.275172 5967 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:46.275256 5967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 05:27:46 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:27:46.304128 5967 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:46.307082 6014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:46.307338 5967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 05:27:46.307425 5967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "ff64417c158b45eebaff0af97b8703dd"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "361e97d39ef272fa6760897aaca15f3a"
server_key_iv: "517c12c416e86aebacd81b1a8556d01f"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:46.307925 5967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:46.328095 5967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:46.330613 5967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:46.330824 5967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:46.331099 5967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:46.331512 5967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:46.331569 5967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:46.331614 5967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:46.331640 5967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:46.339452 5967 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:46407
I20260812 05:27:46.339965 5967 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:27:46.340600 6135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:46407 every 8 connection(s)
Aug 12 05:27:46 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:46.347769 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 5967
I20260812 05:27:46.348098 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:36543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:45437
--local_ip_for_outbound_sockets=127.24.66.66
--tserver_master_addrs=127.24.66.126:44667
--webserver_port=35745
--webserver_interface=127.24.66.66
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.66.84:36543/kms with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:27:46.353941 6138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:46.340924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:44667 (local address 127.24.66.65:36805)
0812 05:27:46.341116 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:46.341154 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:46.341762 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:46.342248 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:46.342252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:46.342516 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:46.342781 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:46.342789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.344110 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.344112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:46.344935 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:46.344937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.345629 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.345631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:46.345770 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:46.346461 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:46.346470 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:46.348246 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:46.351379 (+ 3133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:46.351389 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:46.351391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:46.351767 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:46.352354 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:46.352355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:46.352355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:46.352514 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:46.353092 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:46.353101 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:46.353290 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 05:27:46.353420 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 05:27:46.353628 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":112,"threads_started":1}
I20260812 05:27:46.354602 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:46.341284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:36805 (local address 127.24.66.126:44667)
0812 05:27:46.341543 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:46.341552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:46.341598 (+ 46us) server_negotiation.cc:407] Connection header received
0812 05:27:46.342007 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:46.342012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:46.342060 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:46.342207 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:46.342927 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.343990 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.345043 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.345490 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.345609 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:46.348415 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:46.348420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:46.348421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:46.348436 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:46.351256 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:46.352127 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:46.352131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:46.352132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:46.352263 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:46.352600 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:46.352601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:46.352601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:46.352968 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:46.353059 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:46.354154 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0812 05:27:46.354311 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":101,"threads_started":1}
I20260812 05:27:46.355908 6136 heartbeater.cc:347] Connected to a master server at 127.24.66.126:44667
I20260812 05:27:46.356019 6136 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:46.356261 6136 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
W20260812 05:27:46.357391 6136 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:44667 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 05:27:46.357470 6136 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:46.357549 6136 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
I20260812 05:27:46.358100 6136 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:46.358187 6136 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
W20260812 05:27:46.358626 6136 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:27:46.469950 6140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:46.470309 6140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:46.470476 6140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:46.472499 6140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:46.472572 6140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:46.472617 6140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:27:46.475273 6140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.66.84:36543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:45437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=35745
--tserver_master_addrs=127.24.66.126:44667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:46.475555 6140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:46.475942 6140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:46.481822 6145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:46.481822 6148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:46.482208 6140 server_base.cc:1034] running on GCE node
W20260812 05:27:46.482472 6146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:46.482730 6140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:46.483057 6140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:46.484258 6140 hybrid_clock.cc:648] HybridClock initialized: now 1786512466484242 us; error 31 us; skew 500 ppm
Aug 12 05:27:46 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:46.487273 6140 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:27:46.487816 6140 webserver.cc:460] Webserver started at http://127.24.66.66:35745/ using document root <none> and password file <none>
I20260812 05:27:46.488090 6140 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:46.488183 6140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 05:27:46 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:27:46.505796 6140 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 05:27:46.507117 6157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:46.507318 6140 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:27:46.507419 6140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "1164416641e647be8454e51c859b9b7b"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "fcfd760cb569a0b69591b108565f86c0"
server_key_iv: "d1be3ccd347321343a26392bc4302a89"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:46.507915 6140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:46.522248 6140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:46.524849 6140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:46.525091 6140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:46.525378 6140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:46.525791 6140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:46.525889 6140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:46.525972 6140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:46.526033 6140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:46.532904 6140 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:45437
I20260812 05:27:46.533489 6140 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:27:46.535727 6274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:45437 every 8 connection(s)
I20260812 05:27:46.536217 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 6140
I20260812 05:27:46.536542 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.66.84:36543/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:41623
--local_ip_for_outbound_sockets=127.24.66.67
--tserver_master_addrs=127.24.66.126:44667
--webserver_port=41979
--webserver_interface=127.24.66.67
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.66.84:36543/kms with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:27:46 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:46.550132 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:46.536997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:50157 (local address 127.24.66.126:44667)
0812 05:27:46.537046 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:46.537049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:46.537422 (+ 373us) server_negotiation.cc:407] Connection header received
0812 05:27:46.538303 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:46.538305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:46.538323 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:46.538399 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:46.539405 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.540195 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.542405 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.542722 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.542781 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:46.545854 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:46.545856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:46.545857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:46.545865 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:46.547761 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:46.548824 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:46.548826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:46.548826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:46.548898 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:46.549263 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:46.549265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:46.549265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:46.549423 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:46.549477 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:46.549913 (+ 436us) server_negotiation.cc:299] Negotiation successful
0812 05:27:46.550016 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:27:46.550879 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:46.536477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:44667 (local address 127.24.66.66:50157)
0812 05:27:46.537257 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:46.537331 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:46.538153 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:46.538641 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:46.538647 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:46.538951 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:46.539291 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:46.539298 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.540334 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.540335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:46.541641 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:46.541644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.542913 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.542914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:46.543074 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:46.543837 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:46.543849 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:46.545716 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:46.547869 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:46.547878 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:46.547881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:46.548249 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:46.548979 (+ 730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:46.548980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:46.548980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:46.549144 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:46.549529 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:46.549536 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:46.549727 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 05:27:46.550409 (+ 682us) client_negotiation.cc:240] Negotiation successful
0812 05:27:46.550622 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":95,"threads_started":1}
I20260812 05:27:46.551568 6275 heartbeater.cc:347] Connected to a master server at 127.24.66.126:44667
I20260812 05:27:46.551685 6275 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:46.551892 6275 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
W20260812 05:27:46.552747 6275 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:44667 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 05:27:46.552821 6275 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:46.552886 6275 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
I20260812 05:27:46.553301 6275 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:46.553390 6275 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
W20260812 05:27:46.553747 6275 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:27:46.658672 6278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:46.658919 6278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:46.658964 6278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:46.660228 6278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:46.660305 6278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:46.660353 6278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:27:46.662010 6278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.66.84:36543/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:41623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=41979
--tserver_master_addrs=127.24.66.126:44667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:46.662294 6278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:46.662564 6278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:46.668232 6285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:46.668232 6287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:46.669003 6278 server_base.cc:1034] running on GCE node
W20260812 05:27:46.670552 6284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:46.670855 6278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:46.671175 6278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:46.672338 6278 hybrid_clock.cc:648] HybridClock initialized: now 1786512466672317 us; error 38 us; skew 500 ppm
Aug 12 05:27:46 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:46.675137 6278 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:27:46.675701 6278 webserver.cc:460] Webserver started at http://127.24.66.67:41979/ using document root <none> and password file <none>
I20260812 05:27:46.675976 6278 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:46.676064 6278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 05:27:46 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:27:46.700023 6278 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260812 05:27:46.710142 6296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:46.710631 6278 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.001s
I20260812 05:27:46.710752 6278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "39d7bc7d30f4441a95b70fb4d278f465"
format_stamp: "Formatted at 2026-08-12 05:27:45 on dist-test-slave-zpfg"
server_key: "3c0ae0e860cf43fbe774ae427eb27e9b"
server_key_iv: "21420f97c156d86390f241db40229b95"
server_key_version: "kuduclusterkey@0"
I20260812 05:27:46.711431 6278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:46.728847 6278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:46.731614 6278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:46.731842 6278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:46.732232 6278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:46.732667 6278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:46.732758 6278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:46.732831 6278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:46.732892 6278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:46.740258 6278 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:41623
I20260812 05:27:46.740345 6418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:41623 every 8 connection(s)
I20260812 05:27:46.740875 6278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:27:46.749989 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 6278
Aug 12 05:27:46 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512466, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:46.757946 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:46.742809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:42727 (local address 127.24.66.126:44667)
0812 05:27:46.742859 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:46.742861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:46.746695 (+ 3834us) server_negotiation.cc:407] Connection header received
0812 05:27:46.747380 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:46.747382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:46.747397 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:46.747436 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:46.748356 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.749130 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.751265 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.751573 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.751666 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:46.754409 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:46.754411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:46.754411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:46.754419 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:46.756249 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:46.756788 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:46.756789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:46.756790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:46.756855 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:46.757164 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:46.757165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:46.757166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:46.757311 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:46.757376 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:46.757751 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 05:27:46.757843 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:27:46.758842 6421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:46.742510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:44667 (local address 127.24.66.67:42727)
0812 05:27:46.746496 (+ 3986us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:46.746555 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:46.747237 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:46.747645 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:46.747649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:46.747912 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:46.748220 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:46.748227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.749252 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.749253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:46.751137 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:46.751140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.751702 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.751703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:46.751851 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:46.752494 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:46.752505 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:46.754250 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:46.756354 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:46.756363 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:46.756365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:46.756689 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:46.756926 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:46.756927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:46.756927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:46.757071 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:46.757415 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:46.757422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:46.757593 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 05:27:46.758405 (+ 812us) client_negotiation.cc:240] Negotiation successful
0812 05:27:46.758595 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3930,"thread_start_us":86,"threads_started":1}
I20260812 05:27:46.759653 6419 heartbeater.cc:347] Connected to a master server at 127.24.66.126:44667
I20260812 05:27:46.759754 6419 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:46.759951 6419 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
W20260812 05:27:46.760646 6419 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:44667 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 05:27:46.760718 6419 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:46.760782 6419 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
I20260812 05:27:46.761132 6419 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:46.761224 6419 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
W20260812 05:27:46.761539 6419 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 05:27:46.770876 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:46.750488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60052 (local address 127.24.66.126:44667)
0812 05:27:46.758375 (+ 7887us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:46.758377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:46.762567 (+ 4190us) server_negotiation.cc:407] Connection header received
0812 05:27:46.762673 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:46.762674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:46.762684 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:46.762713 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:46.763659 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.764443 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.765640 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:46.765957 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:46.766039 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:46.767642 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:46.767644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:46.767644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:46.767652 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:46.769415 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:46.769840 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:46.769841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:46.769842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:46.769910 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:46.770118 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:46.770119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:46.770120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:46.770267 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:46.770511 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:46.770557 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 05:27:46.770641 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7837,"thread_start_us":68,"threads_started":1}
Aug 12 05:27:47 dist-test-slave-zpfg krb5kdc[4886](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786512467, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:47.358819 6136 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:47.358996 6136 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
I20260812 05:27:47.359436 5927 ts_manager.cc:194] Registered new tserver with Master: ff64417c158b45eebaff0af97b8703dd (127.24.66.65:46407)
I20260812 05:27:47.360440 5927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:36805
I20260812 05:27:47.553920 6275 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:47.554113 6275 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
I20260812 05:27:47.554589 5927 ts_manager.cc:194] Registered new tserver with Master: 1164416641e647be8454e51c859b9b7b (127.24.66.66:45437)
I20260812 05:27:47.555011 5927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:50157
Aug 12 05:27:47 dist-test-slave-zpfg krb5kdc[4886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512467, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for HTTP/127.24.66.84@KRBTEST.COM
I20260812 05:27:47.761721 6419 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:47.761873 6419 heartbeater.cc:511] Master 127.24.66.126:44667 requested a full tablet report, sending...
I20260812 05:27:47.762228 5927 ts_manager.cc:194] Registered new tserver with Master: 39d7bc7d30f4441a95b70fb4d278f465 (127.24.66.67:41623)
I20260812 05:27:47.762764 5927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:42727
I20260812 05:27:47.771170 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:27:47.801690 6440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.772246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57092 (local address 127.24.66.126:44667)
0812 05:27:47.778948 (+ 6702us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:47.778951 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:47.779131 (+ 180us) server_negotiation.cc:407] Connection header received
0812 05:27:47.779218 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:47.779219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:47.779235 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:47.779316 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:47.780896 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.781647 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.782904 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.783201 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.783273 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.792631 (+ 9358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:47.792634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:47.792634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:47.792647 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:47.799030 (+ 6383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:47.799519 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:47.799521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:47.799521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:47.799800 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:47.800058 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:47.800060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:47.800255 (+ 195us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:47.800435 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:47.800792 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:47.801270 (+ 478us) server_negotiation.cc:299] Negotiation successful
0812 05:27:47.801376 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6668,"thread_start_us":68,"threads_started":1}
I20260812 05:27:47.805053 5927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:27:47.807842 5927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:27:47.818405 6447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.811913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:41623 (local address 127.0.0.1:37804)
0812 05:27:47.812120 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:47.812207 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:47.812290 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:47.813750 (+ 1460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:47.813754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:47.813757 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:47.813845 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:47.813848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.815536 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.815537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:47.816627 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:47.816630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.818050 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.818051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:47.818137 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.818151 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 05:27:47.818206 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 05:27:47.818267 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260812 05:27:47.818645 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.812952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.24.66.66:45437)
0812 05:27:47.813090 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:47.813095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:47.813362 (+ 267us) server_negotiation.cc:407] Connection header received
0812 05:27:47.813432 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:47.813435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:47.813453 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:47.813535 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:47.813971 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.815185 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.817121 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.818035 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.818098 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.818166 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:47.818449 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 05:27:47.818498 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":87,"threads_started":1}
I20260812 05:27:47.820346 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.812199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:45437 (local address 127.0.0.1:33848)
0812 05:27:47.813183 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:47.813314 (+ 131us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:47.813366 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:47.813816 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:47.813816 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:47.813818 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:47.813887 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:47.813888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.815267 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.815268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:47.817031 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:47.817034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.818220 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.818221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:47.818272 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.818275 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:27:47.818372 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 05:27:47.818405 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"mutex_wait_us":4,"thread_start_us":80,"threads_started":1}
I20260812 05:27:47.821075 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.812624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37804 (local address 127.24.66.67:41623)
0812 05:27:47.813483 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:47.813488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:47.813497 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:27:47.813569 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:47.813571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:47.813605 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:47.813946 (+ 341us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:47.814035 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.815441 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.816746 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.817897 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.820812 (+ 2915us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.820896 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:47.820922 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 05:27:47.820970 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":69,"threads_started":1}
I20260812 05:27:47.821272 6208 tablet_service.cc:1467] Processing CreateTablet for tablet d2da204bc6b8455bbaadf4666cc937a3 (DEFAULT_TABLE table=test-table [id=f39c600c881d4092be6b39172035fd59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:27:47.821770 6208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2da204bc6b8455bbaadf4666cc937a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:47.821805 6349 tablet_service.cc:1467] Processing CreateTablet for tablet d2da204bc6b8455bbaadf4666cc937a3 (DEFAULT_TABLE table=test-table [id=f39c600c881d4092be6b39172035fd59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:27:47.822175 6349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2da204bc6b8455bbaadf4666cc937a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:47.822670 6447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.812422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:46407 (local address 127.0.0.1:44098)
0812 05:27:47.818507 (+ 6085us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:47.818519 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:47.818563 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:47.819091 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:47.819092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:47.819094 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:47.819133 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:47.819135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.819946 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.819947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:47.820991 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:47.820994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.822444 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.822446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:47.822514 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.822519 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:27:47.822551 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 05:27:47.822594 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6074,"thread_start_us":124,"threads_started":1}
I20260812 05:27:47.823017 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.812566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44098 (local address 127.24.66.65:46407)
0812 05:27:47.812886 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:47.812892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:47.818915 (+ 6023us) server_negotiation.cc:407] Connection header received
0812 05:27:47.818962 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:47.818964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:47.818981 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:47.819098 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:47.819199 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.819853 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.821361 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.822298 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.822781 (+ 483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.822868 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:47.822897 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 05:27:47.822943 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":85,"threads_started":1}
I20260812 05:27:47.831696 6067 tablet_service.cc:1467] Processing CreateTablet for tablet d2da204bc6b8455bbaadf4666cc937a3 (DEFAULT_TABLE table=test-table [id=f39c600c881d4092be6b39172035fd59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:27:47.832211 6067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2da204bc6b8455bbaadf4666cc937a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:47.832181 6453 tablet_bootstrap.cc:492] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b: Bootstrap starting.
I20260812 05:27:47.833081 6453 tablet_bootstrap.cc:654] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:47.836663 6455 tablet_bootstrap.cc:492] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd: Bootstrap starting.
I20260812 05:27:47.837291 6453 log.cc:824] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:47.837495 6455 tablet_bootstrap.cc:654] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:47.837862 6455 log.cc:824] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:47.839329 6453 tablet_bootstrap.cc:492] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b: No bootstrap required, opened a new log
I20260812 05:27:47.839479 6453 ts_tablet_manager.cc:1397] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260812 05:27:47.839574 6455 tablet_bootstrap.cc:492] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd: No bootstrap required, opened a new log
I20260812 05:27:47.839658 6455 ts_tablet_manager.cc:1397] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:27:47.843214 6455 raft_consensus.cc:348] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d7bc7d30f4441a95b70fb4d278f465" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41623 } } peers { permanent_uuid: "ff64417c158b45eebaff0af97b8703dd" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46407 } } peers { permanent_uuid: "1164416641e647be8454e51c859b9b7b" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45437 } }
I20260812 05:27:47.843436 6455 raft_consensus.cc:374] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:47.843494 6455 raft_consensus.cc:729] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff64417c158b45eebaff0af97b8703dd, State: Initialized, Role: FOLLOWER
I20260812 05:27:47.843655 6455 consensus_queue.cc:260] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d7bc7d30f4441a95b70fb4d278f465" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41623 } } peers { permanent_uuid: "ff64417c158b45eebaff0af97b8703dd" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46407 } } peers { permanent_uuid: "1164416641e647be8454e51c859b9b7b" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45437 } }
I20260812 05:27:47.844003 6455 ts_tablet_manager.cc:1428] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 05:27:47.844151 6136 heartbeater.cc:503] Master 127.24.66.126:44667 was elected leader, sending a full tablet report...
I20260812 05:27:47.850178 6453 raft_consensus.cc:348] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d7bc7d30f4441a95b70fb4d278f465" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41623 } } peers { permanent_uuid: "ff64417c158b45eebaff0af97b8703dd" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46407 } } peers { permanent_uuid: "1164416641e647be8454e51c859b9b7b" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45437 } }
I20260812 05:27:47.850555 6453 raft_consensus.cc:374] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:47.850574 6454 tablet_bootstrap.cc:492] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465: Bootstrap starting.
I20260812 05:27:47.850611 6453 raft_consensus.cc:729] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1164416641e647be8454e51c859b9b7b, State: Initialized, Role: FOLLOWER
I20260812 05:27:47.850811 6453 consensus_queue.cc:260] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d7bc7d30f4441a95b70fb4d278f465" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41623 } } peers { permanent_uuid: "ff64417c158b45eebaff0af97b8703dd" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46407 } } peers { permanent_uuid: "1164416641e647be8454e51c859b9b7b" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45437 } }
I20260812 05:27:47.851233 6453 ts_tablet_manager.cc:1428] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b: Time spent starting tablet: real 0.012s user 0.001s sys 0.004s
I20260812 05:27:47.851424 6275 heartbeater.cc:503] Master 127.24.66.126:44667 was elected leader, sending a full tablet report...
I20260812 05:27:47.851547 6454 tablet_bootstrap.cc:654] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:47.851933 6454 log.cc:824] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:47.853621 6454 tablet_bootstrap.cc:492] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465: No bootstrap required, opened a new log
I20260812 05:27:47.853713 6454 ts_tablet_manager.cc:1397] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:27:47.857877 6454 raft_consensus.cc:348] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d7bc7d30f4441a95b70fb4d278f465" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41623 } } peers { permanent_uuid: "ff64417c158b45eebaff0af97b8703dd" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46407 } } peers { permanent_uuid: "1164416641e647be8454e51c859b9b7b" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45437 } }
I20260812 05:27:47.858129 6454 raft_consensus.cc:374] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:47.858181 6454 raft_consensus.cc:729] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d7bc7d30f4441a95b70fb4d278f465, State: Initialized, Role: FOLLOWER
I20260812 05:27:47.858382 6454 consensus_queue.cc:260] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d7bc7d30f4441a95b70fb4d278f465" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41623 } } peers { permanent_uuid: "ff64417c158b45eebaff0af97b8703dd" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46407 } } peers { permanent_uuid: "1164416641e647be8454e51c859b9b7b" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45437 } }
I20260812 05:27:47.858808 6454 ts_tablet_manager.cc:1428] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 05:27:47.858969 6419 heartbeater.cc:503] Master 127.24.66.126:44667 was elected leader, sending a full tablet report...
I20260812 05:27:47.920483 6458 raft_consensus.cc:482] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:27:47.920660 6458 raft_consensus.cc:504] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d7bc7d30f4441a95b70fb4d278f465" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41623 } } peers { permanent_uuid: "ff64417c158b45eebaff0af97b8703dd" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46407 } } peers { permanent_uuid: "1164416641e647be8454e51c859b9b7b" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45437 } }
I20260812 05:27:47.921185 6458 leader_election.cc:288] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39d7bc7d30f4441a95b70fb4d278f465 (127.24.66.67:41623), 1164416641e647be8454e51c859b9b7b (127.24.66.66:45437)
I20260812 05:27:47.927165 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.922147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:36191 (local address 127.24.66.67:41623)
0812 05:27:47.922189 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:47.922191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:47.922407 (+ 216us) server_negotiation.cc:407] Connection header received
0812 05:27:47.922515 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:47.922517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:47.922526 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:47.922555 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:47.922942 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.923668 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.925679 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.926328 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.926408 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.926418 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:47.927012 (+ 594us) server_negotiation.cc:299] Negotiation successful
0812 05:27:47.927069 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:27:47.927562 6463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.921458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:41623 (local address 127.24.66.65:36191)
0812 05:27:47.922308 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:47.922320 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:47.922440 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:47.922623 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:47.922624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:47.922626 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:47.922850 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:47.922852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.923785 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.923786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:47.925547 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:47.925555 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.926541 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.926542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:47.926665 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.926669 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:27:47.927326 (+ 657us) client_negotiation.cc:240] Negotiation successful
0812 05:27:47.927379 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":189,"threads_started":1}
I20260812 05:27:47.928247 6370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2da204bc6b8455bbaadf4666cc937a3" candidate_uuid: "ff64417c158b45eebaff0af97b8703dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d7bc7d30f4441a95b70fb4d278f465" is_pre_election: true
I20260812 05:27:47.928642 6370 raft_consensus.cc:2454] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff64417c158b45eebaff0af97b8703dd in term 0.
I20260812 05:27:47.929270 6018 leader_election.cc:302] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d7bc7d30f4441a95b70fb4d278f465, ff64417c158b45eebaff0af97b8703dd; no voters:
I20260812 05:27:47.929528 6458 raft_consensus.cc:2781] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:27:47.929625 6458 raft_consensus.cc:482] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:27:47.929682 6458 raft_consensus.cc:3037] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:47.931416 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.922035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:47777 (local address 127.24.66.66:45437)
0812 05:27:47.922094 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:47.922096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:47.926937 (+ 4841us) server_negotiation.cc:407] Connection header received
0812 05:27:47.927927 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:47.927929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:47.927941 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:47.928001 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:47.928194 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.928781 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.930247 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.930933 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.930991 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.931001 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:47.931297 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 05:27:47.931344 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:27:47.931355 6458 raft_consensus.cc:504] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d7bc7d30f4441a95b70fb4d278f465" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41623 } } peers { permanent_uuid: "ff64417c158b45eebaff0af97b8703dd" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46407 } } peers { permanent_uuid: "1164416641e647be8454e51c859b9b7b" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45437 } }
I20260812 05:27:47.931579 6458 leader_election.cc:288] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [CANDIDATE]: Term 1 election: Requested vote from peers 39d7bc7d30f4441a95b70fb4d278f465 (127.24.66.67:41623), 1164416641e647be8454e51c859b9b7b (127.24.66.66:45437)
I20260812 05:27:47.931757 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.921763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:45437 (local address 127.24.66.65:47777)
0812 05:27:47.926866 (+ 5103us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:47.926878 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:47.927813 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:47.928024 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:47.928026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:47.928028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:47.928135 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:47.928136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.928888 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.928889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:47.930120 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:47.930124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.931068 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.931068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:47.931205 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.931209 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:27:47.931560 (+ 351us) client_negotiation.cc:240] Negotiation successful
0812 05:27:47.931616 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5049,"thread_start_us":115,"threads_started":1}
I20260812 05:27:47.931919 6370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2da204bc6b8455bbaadf4666cc937a3" candidate_uuid: "ff64417c158b45eebaff0af97b8703dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d7bc7d30f4441a95b70fb4d278f465"
I20260812 05:27:47.932013 6370 raft_consensus.cc:3037] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:47.932406 6228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2da204bc6b8455bbaadf4666cc937a3" candidate_uuid: "ff64417c158b45eebaff0af97b8703dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1164416641e647be8454e51c859b9b7b" is_pre_election: true
I20260812 05:27:47.932734 6228 raft_consensus.cc:2454] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff64417c158b45eebaff0af97b8703dd in term 0.
I20260812 05:27:47.933041 6227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2da204bc6b8455bbaadf4666cc937a3" candidate_uuid: "ff64417c158b45eebaff0af97b8703dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1164416641e647be8454e51c859b9b7b"
I20260812 05:27:47.933166 6227 raft_consensus.cc:3037] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:47.933274 6370 raft_consensus.cc:2454] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff64417c158b45eebaff0af97b8703dd in term 1.
I20260812 05:27:47.934935 6227 raft_consensus.cc:2454] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff64417c158b45eebaff0af97b8703dd in term 1.
I20260812 05:27:47.935241 6018 leader_election.cc:302] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d7bc7d30f4441a95b70fb4d278f465, ff64417c158b45eebaff0af97b8703dd; no voters:
I20260812 05:27:47.935539 6458 raft_consensus.cc:2781] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:27:47.935848 6458 raft_consensus.cc:686] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [term 1 LEADER]: Becoming Leader. State: Replica: ff64417c158b45eebaff0af97b8703dd, State: Running, Role: LEADER
I20260812 05:27:47.936041 6458 consensus_queue.cc:237] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d7bc7d30f4441a95b70fb4d278f465" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41623 } } peers { permanent_uuid: "ff64417c158b45eebaff0af97b8703dd" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46407 } } peers { permanent_uuid: "1164416641e647be8454e51c859b9b7b" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45437 } }
I20260812 05:27:47.937184 5927 catalog_manager.cc:5495] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd reported cstate change: term changed from 0 to 1, leader changed from <none> to ff64417c158b45eebaff0af97b8703dd (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ff64417c158b45eebaff0af97b8703dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d7bc7d30f4441a95b70fb4d278f465" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff64417c158b45eebaff0af97b8703dd" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1164416641e647be8454e51c859b9b7b" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45437 } health_report { overall_health: UNKNOWN } } }
I20260812 05:27:47.972537 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:47.968570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44110 (local address 127.24.66.65:46407)
0812 05:27:47.968617 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:47.968619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:47.968626 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:27:47.968671 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:47.968672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:47.968696 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:47.968774 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:27:47.968938 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.969682 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.970983 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:47.971373 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:47.971461 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:47.971724 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:27:47.971889 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:27:47.972024 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:47.972407 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 05:27:47.972434 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:27:47.975555 6227 raft_consensus.cc:1261] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b [term 1 FOLLOWER]: Refusing update from remote peer ff64417c158b45eebaff0af97b8703dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:27:47.975926 6458 consensus_queue.cc:1035] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1164416641e647be8454e51c859b9b7b" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:27:47.978880 6370 raft_consensus.cc:1261] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465 [term 1 FOLLOWER]: Refusing update from remote peer ff64417c158b45eebaff0af97b8703dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:27:47.979249 6458 consensus_queue.cc:1035] T d2da204bc6b8455bbaadf4666cc937a3 P ff64417c158b45eebaff0af97b8703dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "39d7bc7d30f4441a95b70fb4d278f465" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:27:47.985893 6467 mvcc.cc:205] Tried to move back new op lower bound from 7317555068824338432 to 7317555068667248640. Current Snapshot: MvccSnapshot[applied={T|T < 7317555068824338432}]
I20260812 05:27:47.986223 6468 mvcc.cc:205] Tried to move back new op lower bound from 7317555068824338432 to 7317555068667248640. Current Snapshot: MvccSnapshot[applied={T|T < 7317555068824338432}]
I20260812 05:27:47.990252 6470 mvcc.cc:205] Tried to move back new op lower bound from 7317555068824338432 to 7317555068667248640. Current Snapshot: MvccSnapshot[applied={T|T < 7317555068824338432}]
I20260812 05:27:47.994825 5927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:27:47.995010 5927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:27:47.996130 5927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0c360944d8949dfb030db42e66cda30: Sending DeleteTablet for 3 replicas of tablet d2da204bc6b8455bbaadf4666cc937a3
I20260812 05:27:47.997287 6349 tablet_service.cc:1514] Processing DeleteTablet for tablet d2da204bc6b8455bbaadf4666cc937a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:27:47 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:37804
I20260812 05:27:47.997864 6208 tablet_service.cc:1514] Processing DeleteTablet for tablet d2da204bc6b8455bbaadf4666cc937a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:27:47 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:33848
I20260812 05:27:47.998395 6067 tablet_service.cc:1514] Processing DeleteTablet for tablet d2da204bc6b8455bbaadf4666cc937a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:27:47 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:44098
I20260812 05:27:47.998709 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 5967
I20260812 05:27:48.006511 6477 tablet_replica.cc:330] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465: stopping tablet replica
I20260812 05:27:48.006803 6478 tablet_replica.cc:330] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b: stopping tablet replica
I20260812 05:27:48.007099 6478 raft_consensus.cc:2229] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:27:48.007380 6477 raft_consensus.cc:2229] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:27:48.007688 6478 raft_consensus.cc:2258] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:27:48.007992 6477 raft_consensus.cc:2258] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:27:48.009028 6477 ts_tablet_manager.cc:1905] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:27:48.014628 6478 ts_tablet_manager.cc:1905] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:27:48.015017 6477 ts_tablet_manager.cc:1918] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:27:48.015362 6477 log.cc:1196] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/d2da204bc6b8455bbaadf4666cc937a3
I20260812 05:27:48.016322 6477 ts_tablet_manager.cc:1939] T d2da204bc6b8455bbaadf4666cc937a3 P 39d7bc7d30f4441a95b70fb4d278f465: Deleting consensus metadata
I20260812 05:27:48.017071 5912 catalog_manager.cc:4841] TS 39d7bc7d30f4441a95b70fb4d278f465 (127.24.66.67:41623): tablet d2da204bc6b8455bbaadf4666cc937a3 (table test-table [id=f39c600c881d4092be6b39172035fd59]) successfully deleted
I20260812 05:27:48.019263 6478 ts_tablet_manager.cc:1918] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:27:48.019383 6478 log.cc:1196] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/d2da204bc6b8455bbaadf4666cc937a3
I20260812 05:27:48.019687 6478 ts_tablet_manager.cc:1939] T d2da204bc6b8455bbaadf4666cc937a3 P 1164416641e647be8454e51c859b9b7b: Deleting consensus metadata
I20260812 05:27:48.020226 5912 catalog_manager.cc:4841] TS 1164416641e647be8454e51c859b9b7b (127.24.66.66:45437): tablet d2da204bc6b8455bbaadf4666cc937a3 (table test-table [id=f39c600c881d4092be6b39172035fd59]) successfully deleted
W20260812 05:27:48.024127 5910 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:46407 (error 108)
I20260812 05:27:48.025192 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:48.025024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:46407 (local address 127.0.0.1:44120)
0812 05:27:48.025101 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:48.025147 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:46407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 05:27:48.025328 5910 catalog_manager.cc:4584] TS ff64417c158b45eebaff0af97b8703dd (127.24.66.65:46407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d2da204bc6b8455bbaadf4666cc937a3: Network error: Client connection negotiation failed: client connection to 127.24.66.65:46407: connect: Connection refused (error 111)
I20260812 05:27:48.025549 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 6140
I20260812 05:27:48.035408 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 6278
I20260812 05:27:48.044607 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 5890
I20260812 05:27:48.054394 24841 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 05:27:48.400465 24841 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 05:27:48.400565 24841 mini_ranger.cc:66] Stopping Ranger...
I20260812 05:27:48.757135 24841 mini_ranger.cc:68] Stopped Ranger
2026-08-12 05:27:48.757 UTC [4917] LOG: received smart shutdown request
2026-08-12 05:27:48.761 UTC [4917] LOG: background worker "logical replication launcher" (PID 4926) exited with exit code 1
2026-08-12 05:27:48.761 UTC [4921] LOG: shutting down
2026-08-12 05:27:48.958 UTC [4917] LOG: database system is shut down
2026-08-12T05:27:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53407 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:27:49 dist-test-slave-zpfg krb5kdc[6492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:27:49 dist-test-slave-zpfg krb5kdc[6492](info): set up 2 sockets
Aug 12 05:27:49 dist-test-slave-zpfg krb5kdc[6492](info): commencing operation
krb5kdc: starting...
W20260812 05:27:50.231292 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:27:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:27:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:27:50.370425 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40049
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:34369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:40049,127.24.66.125:44115,127.24.66.124:38763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:50.463950 6505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:50.464250 6505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:50.464295 6505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:50.465713 6505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:27:50.465802 6505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:50.465830 6505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:50.465855 6505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:27:50.465889 6505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:27:50.467682 6505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:40049,127.24.66.125:44115,127.24.66.124:38763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:50.468021 6505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:50.468297 6505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:50.474320 6510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:50.474453 6513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:50.474524 6511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:50.474977 6505 server_base.cc:1034] running on GCE node
I20260812 05:27:50.475209 6505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:50.475530 6505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:50.476696 6505 hybrid_clock.cc:648] HybridClock initialized: now 1786512470476672 us; error 45 us; skew 500 ppm
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:50.479641 6505 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:27:50.480170 6505 webserver.cc:460] Webserver started at http://127.24.66.126:38123/ using document root <none> and password file <none>
I20260812 05:27:50.480441 6505 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:50.480526 6505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:50.480681 6505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:27:50.481868 6505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "9459d73a2cac4df1b4d3890d405b9584"
format_stamp: "Formatted at 2026-08-12 05:27:50 on dist-test-slave-zpfg"
server_key: "7436c597325488afee43121dee75572e"
server_key_iv: "fd7842efdf7e53883fa698312e506744"
server_key_version: "encryptionkey@0"
I20260812 05:27:50.482255 6505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "9459d73a2cac4df1b4d3890d405b9584"
format_stamp: "Formatted at 2026-08-12 05:27:50 on dist-test-slave-zpfg"
server_key: "7436c597325488afee43121dee75572e"
server_key_iv: "fd7842efdf7e53883fa698312e506744"
server_key_version: "encryptionkey@0"
I20260812 05:27:50.483884 6505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:27:50.485050 6520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:50.485368 6505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:50.485598 6505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "9459d73a2cac4df1b4d3890d405b9584"
format_stamp: "Formatted at 2026-08-12 05:27:50 on dist-test-slave-zpfg"
server_key: "7436c597325488afee43121dee75572e"
server_key_iv: "fd7842efdf7e53883fa698312e506744"
server_key_version: "encryptionkey@0"
I20260812 05:27:50.485710 6505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:50.496644 6505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:50.499156 6505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:50.499398 6505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:50.503943 6505 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:40049
I20260812 05:27:50.503979 6575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:40049 every 8 connection(s)
I20260812 05:27:50.504382 6505 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:27:50.505429 6576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:50.506958 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 6505
I20260812 05:27:50.507126 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:27:50.507359 24841 external_mini_cluster.cc:1428] Setting key 5e1cefbd187ea285c4693837c45f7d04
I20260812 05:27:50.507687 6576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.66.125@KRBTEST.COM; defaulting to no policy
I20260812 05:27:50.520413 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.508697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40049 (local address 127.0.0.1:37048)
0812 05:27:50.508918 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.508974 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:50.509672 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:50.510042 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:50.510045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:50.510499 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:50.510725 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.510731 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.512516 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.512517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.513325 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.513327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.513935 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.513937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.514042 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.514818 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:50.514831 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:50.516854 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:50.518819 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.518825 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.518827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.519027 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.519293 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.519294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.519294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.519380 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.519757 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:50.519763 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:50.519828 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 05:27:50.519943 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 05:27:50.520190 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":105,"thread_start_us":93,"threads_started":1}
I20260812 05:27:50.520413 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.508892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37048 (local address 127.24.66.126:40049)
0812 05:27:50.509027 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.509032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.509088 (+ 56us) server_negotiation.cc:407] Connection header received
0812 05:27:50.509872 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.509874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.509901 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.510023 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.510840 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.512361 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.513416 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.513818 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.513921 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.516963 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.516971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.516971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.516985 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.518717 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.519126 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.519129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.519130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.519215 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.519447 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.519448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.519448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.519677 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.519767 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.519997 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.520142 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.125@KRBTEST.COM" created.
I20260812 05:27:50.521323 6576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:27:50.521734 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.521573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44115 (local address 127.0.0.1:57898)
0812 05:27:50.521651 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.521707 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:44115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 05:27:50.521936 6525 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.66.125:44115: connect: Connection refused (error 111)
I20260812 05:27:50.522475 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.522330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44115 (local address 127.0.0.1:57900)
0812 05:27:50.522412 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.522451 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:44115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 05:27:50.522630 6576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.125:44115: Network error: Client connection negotiation failed: client connection to 127.24.66.125:44115: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab.
Entry for principal kudu/127.24.66.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab.
I20260812 05:27:50.538141 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.125:44115
--webserver_interface=127.24.66.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab
--principal=kudu/127.24.66.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:34369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:40049,127.24.66.125:44115,127.24.66.124:38763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:27:50.571925 6576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } attempt: 1
I20260812 05:27:50.572510 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.572362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44115 (local address 127.0.0.1:57906)
0812 05:27:50.572430 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.572474 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:44115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 05:27:50.572907 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.572764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44115 (local address 127.0.0.1:57920)
0812 05:27:50.572827 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.572874 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:44115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 05:27:50.573048 6576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.125:44115: Network error: Client connection negotiation failed: client connection to 127.24.66.125:44115: connect: Connection refused (error 111)
W20260812 05:27:50.625461 6583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:50.625694 6583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:50.625717 6583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:50.627023 6583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:27:50.627077 6583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:50.627097 6583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:50.627115 6583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:27:50.627175 6583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:27:50.628785 6583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:40049,127.24.66.125:44115,127.24.66.124:38763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab
--principal=kudu/127.24.66.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.125:44115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.66.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:50.629060 6583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:50.629304 6583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:50.634727 6588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:50.634727 6589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:50.634753 6591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:50.635197 6583 server_base.cc:1034] running on GCE node
I20260812 05:27:50.635568 6583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:50.635980 6583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:50.637292 6583 hybrid_clock.cc:648] HybridClock initialized: now 1786512470637282 us; error 72 us; skew 500 ppm
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:50.640587 6583 init.cc:377] Logged in from keytab as kudu/127.24.66.125@KRBTEST.COM (short username kudu)
I20260812 05:27:50.641101 6583 webserver.cc:460] Webserver started at http://127.24.66.125:45403/ using document root <none> and password file <none>
I20260812 05:27:50.641194 6576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } attempt: 2
I20260812 05:27:50.641520 6583 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:50.641606 6583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:50.641742 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.641513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44115 (local address 127.0.0.1:57936)
0812 05:27:50.641637 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.641703 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:44115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 05:27:50.641775 6583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:27:50.642124 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.642014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44115 (local address 127.0.0.1:57950)
0812 05:27:50.642071 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.642099 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:44115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 05:27:50.642267 6576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.125:44115: Network error: Client connection negotiation failed: client connection to 127.24.66.125:44115: connect: Connection refused (error 111)
I20260812 05:27:50.643088 6583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/data/instance:
uuid: "e39136ae11324c39bf66ba10a2cf48b3"
format_stamp: "Formatted at 2026-08-12 05:27:50 on dist-test-slave-zpfg"
server_key: "d35f6acd2c20f9f5243e207a9e128ce5"
server_key_iv: "7b5fa4c6c2ce9b753789c3ae47622132"
server_key_version: "encryptionkey@0"
I20260812 05:27:50.643498 6583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/wal/instance:
uuid: "e39136ae11324c39bf66ba10a2cf48b3"
format_stamp: "Formatted at 2026-08-12 05:27:50 on dist-test-slave-zpfg"
server_key: "d35f6acd2c20f9f5243e207a9e128ce5"
server_key_iv: "7b5fa4c6c2ce9b753789c3ae47622132"
server_key_version: "encryptionkey@0"
I20260812 05:27:50.644989 6583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:27:50.646067 6598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:50.646456 6583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:50.646570 6583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/wal
uuid: "e39136ae11324c39bf66ba10a2cf48b3"
format_stamp: "Formatted at 2026-08-12 05:27:50 on dist-test-slave-zpfg"
server_key: "d35f6acd2c20f9f5243e207a9e128ce5"
server_key_iv: "7b5fa4c6c2ce9b753789c3ae47622132"
server_key_version: "encryptionkey@0"
I20260812 05:27:50.646644 6583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:50.660319 6583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:50.662976 6583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:50.663283 6583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:50.668051 6583 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.125:44115
I20260812 05:27:50.668085 6653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.125:44115 every 8 connection(s)
I20260812 05:27:50.668486 6583 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
I20260812 05:27:50.669432 6654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:50.671203 6654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:27:50.674563 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 6583
I20260812 05:27:50.674657 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-1/wal/instance
I20260812 05:27:50.674902 24841 external_mini_cluster.cc:1428] Setting key f97540e7060ad3df0e140a50b438a6cf
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:50.681375 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.671957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37060 (local address 127.24.66.126:40049)
0812 05:27:50.672019 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.672021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.672270 (+ 249us) server_negotiation.cc:407] Connection header received
0812 05:27:50.672787 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.672788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.672801 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.672865 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.673665 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.674178 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.675462 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.675816 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.675897 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.678311 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.678312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.678312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.678319 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.679884 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.680326 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.680327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.680328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.680391 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.680647 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.680648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.680648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.680816 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.680897 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.681184 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.681277 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 05:27:50.681576 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.671910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40049 (local address 127.0.0.1:37060)
0812 05:27:50.672111 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.672157 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:50.672655 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:50.673002 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:50.673006 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:50.673347 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:50.673556 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.673562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.674305 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.674306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.675364 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.675368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.675912 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.675913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.676019 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.676731 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:50.676743 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:50.678188 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:50.679990 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.679999 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.680001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.680245 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.680467 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.680468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.680468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.680578 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.680899 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:50.680907 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:50.681052 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 05:27:50.681162 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 05:27:50.681334 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":108,"threads_started":1}
I20260812 05:27:50.682195 6654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.66.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.124@KRBTEST.COM" created.
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:27:50.690944 6658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.682570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57958 (local address 127.24.66.125:44115)
0812 05:27:50.682808 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.682812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.682830 (+ 18us) server_negotiation.cc:407] Connection header received
0812 05:27:50.682905 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.682907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.682919 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.682981 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.683330 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.684092 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.684851 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.685153 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.685214 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.687330 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.687340 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.687341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.687348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.689235 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.689710 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.689713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.689713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.689849 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.690120 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.690121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.690122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.690451 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.690532 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.690740 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.690847 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":210,"thread_start_us":123,"threads_started":1}
I20260812 05:27:50.691125 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.682485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44115 (local address 127.0.0.1:57958)
0812 05:27:50.682770 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.682782 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:50.682841 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:50.683007 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:50.683008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:50.683162 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:50.683227 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.683228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.684192 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.684193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.684761 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.684763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.685230 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.685231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.685306 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.685825 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:50.685830 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:50.687243 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:50.689328 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.689330 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.689330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.689620 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.689930 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.689931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.689932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.690015 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.690535 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:50.690541 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:50.690619 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 05:27:50.690714 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 05:27:50.690943 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":95}
I20260812 05:27:50.692109 6654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:27:50.692703 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.692519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49344)
0812 05:27:50.692583 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.692667 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 05:27:50.692878 6602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
I20260812 05:27:50.693373 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.693240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49360)
0812 05:27:50.693301 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.693338 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 05:27:50.693621 6654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:38763: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab.
Entry for principal kudu/127.24.66.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab.
I20260812 05:27:50.706324 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.124:38763
--webserver_interface=127.24.66.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab
--principal=kudu/127.24.66.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:34369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:40049,127.24.66.125:44115,127.24.66.124:38763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:27:50.733462 6576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } attempt: 3
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:27:50.741976 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.733921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44115 (local address 127.0.0.1:57966)
0812 05:27:50.734026 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.734046 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:50.734134 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:50.734378 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:50.734379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:50.734569 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:50.734673 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.734675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.735400 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.735401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.736007 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.736009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.736464 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.736465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.736561 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.737085 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:50.737090 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:50.738686 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:50.740735 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.740736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.740736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.740963 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.741271 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.741272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.741272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.741331 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.741620 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:50.741621 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:50.741672 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 05:27:50.741747 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 05:27:50.741871 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 05:27:50.741976 6658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.734062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57966 (local address 127.24.66.125:44115)
0812 05:27:50.734214 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.734216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.734226 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:27:50.734261 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.734262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.734273 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.734346 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.734793 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.735301 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.736145 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.736373 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.736454 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.738786 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.738789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.738789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.738799 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.740631 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.741074 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.741076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.741076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.741157 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.741420 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.741421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.741421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.741548 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.741685 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.741780 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.741871 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260812 05:27:50.742548 6576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:27:50.742866 6654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } attempt: 1
I20260812 05:27:50.742945 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.742832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49372)
0812 05:27:50.742886 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.742921 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 05:27:50.743234 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.743103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49386)
0812 05:27:50.743171 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.743209 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 05:27:50.743403 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.743312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49396)
0812 05:27:50.743347 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.743375 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 05:27:50.743551 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.743477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49410)
0812 05:27:50.743507 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.743532 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260812 05:27:50.743541 6576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:38763: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
W20260812 05:27:50.743624 6654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:38763: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
I20260812 05:27:50.774678 6576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } attempt: 1
I20260812 05:27:50.775372 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.775166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49422)
0812 05:27:50.775251 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.775323 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 05:27:50.775853 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.775748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49424)
0812 05:27:50.775777 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.775822 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260812 05:27:50.776013 6576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:38763: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
W20260812 05:27:50.801777 6661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:50.802002 6661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:50.802027 6661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:50.803274 6661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:27:50.803320 6661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:50.803334 6661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:50.803347 6661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:27:50.803360 6661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:27:50.804914 6661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:40049,127.24.66.125:44115,127.24.66.124:38763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab
--principal=kudu/127.24.66.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.124:38763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.66.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:50.805189 6661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:50.805447 6661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:27:50.810894 6661 server_base.cc:1034] running on GCE node
W20260812 05:27:50.810835 6666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:50.810832 6667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:50.810832 6670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:50.811595 6661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:50.811764 6654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } attempt: 2
I20260812 05:27:50.812079 6661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:50.812217 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.812052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49432)
0812 05:27:50.812130 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.812172 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 05:27:50.812640 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.812541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49444)
0812 05:27:50.812591 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.812616 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 05:27:50.812819 6654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:38763: Network error: Client connection negotiation failed: client connection to 127.24.66.124:38763: connect: Connection refused (error 111)
I20260812 05:27:50.813424 6661 hybrid_clock.cc:648] HybridClock initialized: now 1786512470813399 us; error 38 us; skew 500 ppm
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:50.816527 6661 init.cc:377] Logged in from keytab as kudu/127.24.66.124@KRBTEST.COM (short username kudu)
I20260812 05:27:50.817121 6661 webserver.cc:460] Webserver started at http://127.24.66.124:36873/ using document root <none> and password file <none>
I20260812 05:27:50.817376 6661 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:50.817425 6661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:50.817577 6661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:27:50.818819 6661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/data/instance:
uuid: "ea8a3e12f7cb4442994519d4d0fdd94d"
format_stamp: "Formatted at 2026-08-12 05:27:50 on dist-test-slave-zpfg"
server_key: "828b148624f84d2999f05a188917e53f"
server_key_iv: "00acc79f91d0457816e7bd2ccaa42cae"
server_key_version: "encryptionkey@0"
I20260812 05:27:50.819224 6661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/wal/instance:
uuid: "ea8a3e12f7cb4442994519d4d0fdd94d"
format_stamp: "Formatted at 2026-08-12 05:27:50 on dist-test-slave-zpfg"
server_key: "828b148624f84d2999f05a188917e53f"
server_key_iv: "00acc79f91d0457816e7bd2ccaa42cae"
server_key_version: "encryptionkey@0"
I20260812 05:27:50.820711 6661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:27:50.821594 6677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:50.821848 6661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:50.821955 6661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/wal
uuid: "ea8a3e12f7cb4442994519d4d0fdd94d"
format_stamp: "Formatted at 2026-08-12 05:27:50 on dist-test-slave-zpfg"
server_key: "828b148624f84d2999f05a188917e53f"
server_key_iv: "00acc79f91d0457816e7bd2ccaa42cae"
server_key_version: "encryptionkey@0"
I20260812 05:27:50.822036 6661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:50.837074 6661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:50.839756 6661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:50.839910 6661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:50.844261 6661 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.124:38763
I20260812 05:27:50.844338 6732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.124:38763 every 8 connection(s)
I20260812 05:27:50.844698 6661 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
I20260812 05:27:50.845664 6733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:50.847409 6733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:27:50.848569 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 6661
I20260812 05:27:50.848714 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786512301961891-24841-0/minicluster-data/master-2/wal/instance
I20260812 05:27:50.848919 24841 external_mini_cluster.cc:1428] Setting key a8a13eac0ed26703b3da7032a33dcf15
I20260812 05:27:50.851259 6576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } attempt: 2
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
I20260812 05:27:50.861405 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.851529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49464)
0812 05:27:50.851584 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.851595 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:50.851634 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:50.852721 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:50.852723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:50.852941 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:50.852986 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.852988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.853801 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.853802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.854276 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.854277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.855183 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.855184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.855241 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.855692 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:50.855698 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:50.857363 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:50.859873 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.859875 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.859875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.860086 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.860397 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.860398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.860398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.860476 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.860980 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:50.860981 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:50.861034 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 05:27:50.861138 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 05:27:50.861274 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 05:27:50.861582 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.848351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.24.66.126:40049)
0812 05:27:50.848429 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.848431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.848723 (+ 292us) server_negotiation.cc:407] Connection header received
0812 05:27:50.849748 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.849751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.849766 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.849832 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.850669 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.851141 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.852530 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.852906 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.852968 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.857617 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.857619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.857619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.857627 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.859745 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.860254 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.860255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.860255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.860316 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.860613 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.860614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.860614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.860832 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.860932 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.861355 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.861450 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:27:50.861826 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.849384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49456 (local address 127.24.66.124:38763)
0812 05:27:50.849608 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.849613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.849645 (+ 32us) server_negotiation.cc:407] Connection header received
0812 05:27:50.849769 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.849770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.849788 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.849890 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.850667 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.851914 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.852913 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.853368 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.853468 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.856928 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.856941 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.856942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.856950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.859821 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.860382 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.860384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.860384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.860452 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.860727 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.860728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.860728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.860900 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.861073 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.861634 (+ 561us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.861728 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":111,"threads_started":1}
I20260812 05:27:50.861903 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.848314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40049 (local address 127.0.0.1:37074)
0812 05:27:50.848566 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.848613 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:50.849581 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:50.849874 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:50.849879 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:50.850262 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:50.850532 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.850537 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.851231 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.851232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.852431 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.852434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.852987 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.852988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.853173 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.854493 (+ 1320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:50.854503 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:50.857513 (+ 3010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:50.859855 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.859864 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.859867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.860162 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.860392 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.860392 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.860393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.860537 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.860922 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:50.860928 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:50.861015 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 05:27:50.861178 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 05:27:50.861391 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":195,"thread_start_us":155,"threads_started":1}
I20260812 05:27:50.862082 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.851567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49464 (local address 127.24.66.124:38763)
0812 05:27:50.852550 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.852551 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.852557 (+ 6us) server_negotiation.cc:407] Connection header received
0812 05:27:50.852573 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.852573 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.852592 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.852695 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.853073 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.853708 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.854375 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.854658 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.854716 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.857531 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.857532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.857533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.857540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.859737 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.860202 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.860206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.860207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.860313 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.860558 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.860559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.860559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.860870 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.861001 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.861180 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.861981 (+ 801us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":963,"thread_start_us":117,"threads_started":1}
I20260812 05:27:50.862571 6733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:27:50.864260 6576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584: Bootstrap starting.
I20260812 05:27:50.864946 6576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:50.865310 6576 log.cc:824] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:50.866427 6576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584: No bootstrap required, opened a new log
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:27:50.868763 6576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } }
I20260812 05:27:50.868949 6576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:50.869058 6576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9459d73a2cac4df1b4d3890d405b9584, State: Initialized, Role: FOLLOWER
I20260812 05:27:50.869230 6576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } }
I20260812 05:27:50.869608 6742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } } }
I20260812 05:27:50.869771 6742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:27:50.869670 6576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:27:50.870397 6658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.862955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.24.66.125:44115)
0812 05:27:50.863010 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.863012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.863112 (+ 100us) server_negotiation.cc:407] Connection header received
0812 05:27:50.863189 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.863200 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.863211 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.863267 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.863561 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.864048 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.864817 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.865073 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.865137 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.867276 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.867278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.867278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.867287 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.868787 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.869473 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.869474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.869474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.869549 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.869777 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.869778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.869778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.869920 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.869969 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.870190 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.870275 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 05:27:50.870399 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.862839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:44115 (local address 127.0.0.1:57978)
0812 05:27:50.863036 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.863046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:50.863123 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:50.863278 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:50.863279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:50.863426 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:50.863479 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.863481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.864143 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.864144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.864729 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.864732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.865132 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.865132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.865214 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.865817 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:50.865823 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:50.867175 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:50.868894 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.868895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.868895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.869127 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.869631 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.869633 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.869633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.869697 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.870022 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:50.870023 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:50.870071 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 05:27:50.870161 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 05:27:50.870276 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184}
I20260812 05:27:50.871145 6733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
W20260812 05:27:50.872148 6755 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:27:50.872251 6755 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
I20260812 05:27:50.876200 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.862968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37090 (local address 127.24.66.126:40049)
0812 05:27:50.865623 (+ 2655us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.865625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.865634 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:27:50.865658 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.865659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.865670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.865734 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.866585 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.867330 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.868465 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.868784 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.868836 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.872347 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.872349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.872349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.872357 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.874219 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.874789 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.874791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.874791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.874867 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.875184 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.875185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.875185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.875344 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.875481 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.875986 (+ 505us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.876075 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2632}
I20260812 05:27:50.873142 6505 master_runner.cc:186] Error getting master registration for 127.24.66.125:44115: OK, instance_id { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" instance_seqno: 1786512470666679 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:27:50.877009 6505 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 05:27:50.879204 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.871423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49466)
0812 05:27:50.871467 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.871478 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:50.871523 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:50.871675 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:50.871676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:50.871828 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:50.871865 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.871867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.872691 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.872692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.873281 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.873284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.873734 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.873735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.873788 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.874207 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:50.874211 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:50.875721 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:50.877936 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.877936 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.877937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.878145 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.878386 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.878388 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.878388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.878469 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.878848 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:50.878849 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:50.878898 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 05:27:50.878982 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 05:27:50.879096 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 05:27:50.879297 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.871453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49466 (local address 127.24.66.124:38763)
0812 05:27:50.871487 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.871489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.871510 (+ 21us) server_negotiation.cc:407] Connection header received
0812 05:27:50.871595 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.871596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.871606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.871644 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.871949 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.872572 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.873364 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.873642 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.873725 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.875813 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.875814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.875815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.875821 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.877847 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.878222 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.878223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.878223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.878272 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.878550 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.878551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.878551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.878758 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.878835 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.879024 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.879165 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":18}
I20260812 05:27:50.880875 6733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d: Bootstrap starting.
I20260812 05:27:50.881667 6733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d: Neither blocks nor log segments found. Creating new log.
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:27:50.881978 6733 log.cc:824] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:50.882962 6733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d: No bootstrap required, opened a new log
I20260812 05:27:50.885494 6733 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } }
I20260812 05:27:50.885705 6733 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:50.885828 6733 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea8a3e12f7cb4442994519d4d0fdd94d, State: Initialized, Role: FOLLOWER
I20260812 05:27:50.886016 6733 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } }
I20260812 05:27:50.886433 6758 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } } }
I20260812 05:27:50.886535 6733 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:27:50.886680 6658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.876870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57992 (local address 127.24.66.125:44115)
0812 05:27:50.876914 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.876916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.876926 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:27:50.877006 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.877007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.877019 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.877082 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.877651 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.878160 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.879416 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.879765 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.879826 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.882487 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.882489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.882489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.882497 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.884350 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.884792 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.884793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.884793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.884852 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.885046 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.885047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.885047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.885189 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.886024 (+ 835us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.886472 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.886562 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:27:50.886631 6758 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:27:50.888403 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 05:27:50.889678 6770 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:27:50.889743 6770 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:27:50.890697 6661 master_runner.cc:186] Error getting master registration for 127.24.66.125:44115: OK, instance_id { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" instance_seqno: 1786512470666679 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:27:50.893136 6661 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 05:27:50.898088 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.890109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37096 (local address 127.24.66.126:40049)
0812 05:27:50.890421 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.890423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.890431 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:27:50.890452 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.890453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.890462 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.890547 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.891338 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.892053 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.893085 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.893357 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.893396 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.895057 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.895058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.895058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.895067 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.896849 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.897228 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.897230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.897230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.897290 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.897476 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.897477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.897477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.897597 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.897664 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.897885 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.897990 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290}
I20260812 05:27:50.903393 6658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.890369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57998 (local address 127.24.66.125:44115)
0812 05:27:50.890420 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.890422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.894576 (+ 4154us) server_negotiation.cc:407] Connection header received
0812 05:27:50.894687 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.894688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.894701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.894733 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.895693 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.896757 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.897793 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.898076 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.898121 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.900030 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.900032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.900032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.900039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.901686 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.902065 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.902066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.902066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.902121 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.902379 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.902380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.902380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.902829 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.902915 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.903177 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.903283 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:27:50.903999 6654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } attempt: 3
I20260812 05:27:50.906683 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.890965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49478 (local address 127.24.66.124:38763)
0812 05:27:50.891003 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.891005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.898522 (+ 7517us) server_negotiation.cc:407] Connection header received
0812 05:27:50.898641 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.898643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.898656 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.898689 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.899411 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.900743 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.901796 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.902120 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.902161 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.903579 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.903581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.903581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.903627 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.905378 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.905764 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.905765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.905765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.905823 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.906029 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.906030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.906030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.906147 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.906232 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.906490 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.906570 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 12 05:27:50 dist-test-slave-zpfg krb5kdc[6492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512470, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
I20260812 05:27:50.912524 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.904487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49492 (local address 127.24.66.124:38763)
0812 05:27:50.904603 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:50.904605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:50.904614 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:27:50.904800 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:50.904801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:50.904815 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:50.904864 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:50.905185 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.905946 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.907050 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.907427 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.907514 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.909580 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:50.909582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:50.909582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:50.909590 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:50.911269 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.911651 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.911652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.911653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.911709 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:50.911939 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:50.911940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:50.911941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:50.912065 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:50.912163 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:50.912342 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 05:27:50.912425 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 05:27:50.912524 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:50.904292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:38763 (local address 127.0.0.1:49492)
0812 05:27:50.904370 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:50.904384 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:50.904703 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:50.904906 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:50.904907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:50.905057 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:50.905102 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.905103 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.906034 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.906035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.906946 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:50.906949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:50.907526 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:50.907527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:50.907584 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:50.908039 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:50.908044 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:50.909486 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:50.911388 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.911390 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.911390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.911572 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.911805 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:50.911806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:50.911807 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:50.911867 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:50.912155 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:50.912156 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:50.912208 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 05:27:50.912313 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 05:27:50.912426 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 05:27:50.914592 6654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3: Bootstrap starting.
I20260812 05:27:50.915431 6654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:50.915791 6654 log.cc:824] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:50.919239 6654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3: No bootstrap required, opened a new log
I20260812 05:27:50.921399 6654 raft_consensus.cc:348] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } }
I20260812 05:27:50.921572 6654 raft_consensus.cc:374] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:50.921610 6654 raft_consensus.cc:729] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e39136ae11324c39bf66ba10a2cf48b3, State: Initialized, Role: FOLLOWER
I20260812 05:27:50.921758 6654 consensus_queue.cc:260] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } }
I20260812 05:27:50.922130 6654 sys_catalog.cc:564] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 05:27:50.925905 6795 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:27:50.926038 6795 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:27:50.924764 6780 sys_catalog.cc:455] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } } }
I20260812 05:27:50.926599 6780 sys_catalog.cc:458] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:27:50.927475 6583 master_runner.cc:263] No leader master found from master e39136ae11324c39bf66ba10a2cf48b3
I20260812 05:27:50.927523 6583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 05:27:50.940816 6758 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:27:50.940979 6758 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } }
I20260812 05:27:50.941359 6758 leader_election.cc:288] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9459d73a2cac4df1b4d3890d405b9584 (127.24.66.126:40049), e39136ae11324c39bf66ba10a2cf48b3 (127.24.66.125:44115)
I20260812 05:27:50.941722 6550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9459d73a2cac4df1b4d3890d405b9584" is_pre_election: true
I20260812 05:27:50.941896 6550 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea8a3e12f7cb4442994519d4d0fdd94d in term 0.
I20260812 05:27:50.942378 6678 leader_election.cc:302] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9459d73a2cac4df1b4d3890d405b9584, ea8a3e12f7cb4442994519d4d0fdd94d; no voters:
I20260812 05:27:50.942672 6758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:27:50.942739 6758 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:27:50.942768 6758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:50.943102 6627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e39136ae11324c39bf66ba10a2cf48b3" is_pre_election: true
I20260812 05:27:50.943262 6627 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea8a3e12f7cb4442994519d4d0fdd94d in term 0.
I20260812 05:27:50.943800 6758 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } }
I20260812 05:27:50.943923 6758 leader_election.cc:288] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [CANDIDATE]: Term 1 election: Requested vote from peers 9459d73a2cac4df1b4d3890d405b9584 (127.24.66.126:40049), e39136ae11324c39bf66ba10a2cf48b3 (127.24.66.125:44115)
I20260812 05:27:50.954674 6550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9459d73a2cac4df1b4d3890d405b9584"
I20260812 05:27:50.954823 6550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:50.955950 6550 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea8a3e12f7cb4442994519d4d0fdd94d in term 1.
I20260812 05:27:50.956341 6678 leader_election.cc:302] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9459d73a2cac4df1b4d3890d405b9584, ea8a3e12f7cb4442994519d4d0fdd94d; no voters:
I20260812 05:27:50.957609 6758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:27:50.957576 6627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e39136ae11324c39bf66ba10a2cf48b3"
I20260812 05:27:50.957675 6627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:50.957805 6758 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [term 1 LEADER]: Becoming Leader. State: Replica: ea8a3e12f7cb4442994519d4d0fdd94d, State: Running, Role: LEADER
I20260812 05:27:50.957918 6758 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } }
I20260812 05:27:50.958426 6758 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea8a3e12f7cb4442994519d4d0fdd94d. Latest consensus state: current_term: 1 leader_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } } }
I20260812 05:27:50.958516 6758 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:50.958848 6627 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea8a3e12f7cb4442994519d4d0fdd94d in term 1.
I20260812 05:27:50.963922 6800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:27:50.964180 6800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:27:50.966804 6627 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [term 1 FOLLOWER]: Refusing update from remote peer ea8a3e12f7cb4442994519d4d0fdd94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:27:50.967121 6550 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [term 1 FOLLOWER]: Refusing update from remote peer ea8a3e12f7cb4442994519d4d0fdd94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:27:50.967775 6758 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:27:50.967949 6758 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:27:50.969884 6742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea8a3e12f7cb4442994519d4d0fdd94d. Latest consensus state: current_term: 1 leader_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } } }
I20260812 05:27:50.970007 6742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:27:50.973089 6780 sys_catalog.cc:455] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea8a3e12f7cb4442994519d4d0fdd94d. Latest consensus state: current_term: 1 leader_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } } }
I20260812 05:27:50.973193 6780 sys_catalog.cc:458] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:27:50.975178 6758 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } } }
I20260812 05:27:50.975298 6758 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:50.975905 6742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } } }
I20260812 05:27:50.976003 6742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:27:50.976342 6801 mvcc.cc:205] Tried to move back new op lower bound from 7317555081076346880 to 7317555081044406272. Current Snapshot: MvccSnapshot[applied={T|T < 7317555081076346880}]
I20260812 05:27:50.976492 6802 mvcc.cc:205] Tried to move back new op lower bound from 7317555081076346880 to 7317555081044406272. Current Snapshot: MvccSnapshot[applied={T|T < 7317555081076346880}]
I20260812 05:27:50.977949 6758 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } } }
I20260812 05:27:50.978044 6758 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:50.978677 6780 sys_catalog.cc:455] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9459d73a2cac4df1b4d3890d405b9584" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40049 } } peers { permanent_uuid: "e39136ae11324c39bf66ba10a2cf48b3" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 44115 } } peers { permanent_uuid: "ea8a3e12f7cb4442994519d4d0fdd94d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 38763 } } }
I20260812 05:27:50.978771 6780 sys_catalog.cc:458] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:27:50.979080 6800 catalog_manager.cc:1347] Generated new cluster ID: 507fd9e8c0bd4942a80f00e4a28a14f8
I20260812 05:27:50.979172 6800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:27:50.986462 6804 mvcc.cc:205] Tried to move back new op lower bound from 7317555081076346880 to 7317555081044406272. Current Snapshot: MvccSnapshot[applied={T|T < 7317555081076346880 or (T in {7317555081076346880})}]
I20260812 05:27:51.004945 6800 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:27:51.005856 6800 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:27:51.016217 6800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea8a3e12f7cb4442994519d4d0fdd94d: Generated new TSK 0
I20260812 05:27:51.016554 6800 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:27:51.872748 6755 catalog_manager.cc:1259] Loaded cluster ID: 507fd9e8c0bd4942a80f00e4a28a14f8
I20260812 05:27:51.872862 6755 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584: loading cluster ID for follower catalog manager: success
I20260812 05:27:51.873718 6755 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584: acquiring CA information for follower catalog manager: success
I20260812 05:27:51.874065 6755 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9459d73a2cac4df1b4d3890d405b9584: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:27:51.926617 6795 catalog_manager.cc:1259] Loaded cluster ID: 507fd9e8c0bd4942a80f00e4a28a14f8
I20260812 05:27:51.926709 6795 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3: loading cluster ID for follower catalog manager: success
I20260812 05:27:51.928035 6795 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3: acquiring CA information for follower catalog manager: success
I20260812 05:27:51.928481 6795 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e39136ae11324c39bf66ba10a2cf48b3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:27:52.073884 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 6505
I20260812 05:27:52.080958 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 6583
I20260812 05:27:52.088419 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 6661
2026-08-12T05:27:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3013 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:27:52 dist-test-slave-zpfg krb5kdc[6819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:27:52 dist-test-slave-zpfg krb5kdc[6819](info): set up 2 sockets
Aug 12 05:27:52 dist-test-slave-zpfg krb5kdc[6819](info): commencing operation
krb5kdc: starting...
W20260812 05:27:53.187994 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:27:53 dist-test-slave-zpfg krb5kdc[6819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:27:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:27:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:27:53.326649 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:32845
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:32845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:53.412503 6833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:53.412806 6833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:53.412849 6833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:53.414122 6833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:27:53.414199 6833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:53.414224 6833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:53.414258 6833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:27:53.414294 6833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:27:53.415962 6833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:32845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:32845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:53.416297 6833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:53.416559 6833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:53.422240 6843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:53.422278 6839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:53.422447 6841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:53.422583 6833 server_base.cc:1034] running on GCE node
I20260812 05:27:53.422951 6833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:53.423234 6833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:53.424423 6833 hybrid_clock.cc:648] HybridClock initialized: now 1786512473424412 us; error 35 us; skew 500 ppm
Aug 12 05:27:53 dist-test-slave-zpfg krb5kdc[6819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512473, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:53.427474 6833 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:27:53.428037 6833 webserver.cc:460] Webserver started at http://127.24.66.126:43037/ using document root <none> and password file <none>
I20260812 05:27:53.428320 6833 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:53.428395 6833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:53.428565 6833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:27:53.429731 6833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "5a33ae696ada4f748d928ec807971d48"
format_stamp: "Formatted at 2026-08-12 05:27:53 on dist-test-slave-zpfg"
server_key: "3ed985b02eafd1c87c789ae37d463c77"
server_key_iv: "6f6a008abe786d7865c4876577879c2a"
server_key_version: "encryptionkey@0"
I20260812 05:27:53.430130 6833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "5a33ae696ada4f748d928ec807971d48"
format_stamp: "Formatted at 2026-08-12 05:27:53 on dist-test-slave-zpfg"
server_key: "3ed985b02eafd1c87c789ae37d463c77"
server_key_iv: "6f6a008abe786d7865c4876577879c2a"
server_key_version: "encryptionkey@0"
I20260812 05:27:53.431697 6833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:27:53.432685 6851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:53.432910 6833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:53.433025 6833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "5a33ae696ada4f748d928ec807971d48"
format_stamp: "Formatted at 2026-08-12 05:27:53 on dist-test-slave-zpfg"
server_key: "3ed985b02eafd1c87c789ae37d463c77"
server_key_iv: "6f6a008abe786d7865c4876577879c2a"
server_key_version: "encryptionkey@0"
I20260812 05:27:53.433105 6833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:53.443696 6833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:53.446115 6833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:53.446306 6833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:53.451022 6833 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:32845
I20260812 05:27:53.451092 6908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:32845 every 8 connection(s)
I20260812 05:27:53.451478 6833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:27:53.452508 6909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:53.452857 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 6833
I20260812 05:27:53.452948 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:27:53.453177 24841 external_mini_cluster.cc:1428] Setting key 14f3af9a0485fbe25652b0c9576c165d
I20260812 05:27:53.457160 6909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48: Bootstrap starting.
I20260812 05:27:53.457864 6909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:53.458211 6909 log.cc:824] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:53.459107 6909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48: No bootstrap required, opened a new log
Aug 12 05:27:53 dist-test-slave-zpfg krb5kdc[6819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:53.461213 6909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33ae696ada4f748d928ec807971d48" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32845 } }
I20260812 05:27:53.461396 6909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:53.461437 6909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a33ae696ada4f748d928ec807971d48, State: Initialized, Role: FOLLOWER
I20260812 05:27:53.461606 6909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33ae696ada4f748d928ec807971d48" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32845 } }
I20260812 05:27:53.461737 6909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:27:53.461782 6909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:27:53.461848 6909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:53.462720 6909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33ae696ada4f748d928ec807971d48" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32845 } }
I20260812 05:27:53.462849 6909 leader_election.cc:302] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a33ae696ada4f748d928ec807971d48; no voters:
I20260812 05:27:53.463042 6909 leader_election.cc:288] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:27:53.463105 6915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:27:53.463313 6915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [term 1 LEADER]: Becoming Leader. State: Replica: 5a33ae696ada4f748d928ec807971d48, State: Running, Role: LEADER
I20260812 05:27:53.463331 6909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:27:53.463482 6915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33ae696ada4f748d928ec807971d48" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32845 } }
I20260812 05:27:53.463851 6917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a33ae696ada4f748d928ec807971d48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33ae696ada4f748d928ec807971d48" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32845 } } }
I20260812 05:27:53.463953 6917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:53.464226 6915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a33ae696ada4f748d928ec807971d48. Latest consensus state: current_term: 1 leader_uuid: "5a33ae696ada4f748d928ec807971d48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a33ae696ada4f748d928ec807971d48" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 32845 } } }
I20260812 05:27:53.464316 6915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48 [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:53.464643 6920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:27:53.465253 6920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:27:53.465667 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:53.453841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56084 (local address 127.24.66.126:32845)
0812 05:27:53.454066 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:53.454078 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:53.454121 (+ 43us) server_negotiation.cc:407] Connection header received
0812 05:27:53.454752 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:53.454763 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:53.454839 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:53.455012 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:53.455735 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:53.457383 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:53.458204 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:53.458790 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:53.458917 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:53.461431 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:53.461436 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:53.461437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:53.461451 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:53.463763 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:53.464216 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:53.464220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:53.464221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:53.464368 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:53.464608 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:53.464609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:53.464609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:53.464974 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:53.465128 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:53.465293 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 05:27:53.465455 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":88,"threads_started":1}
I20260812 05:27:53.466909 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:27:53.479693 6920 catalog_manager.cc:1347] Generated new cluster ID: 9db75b7f419e499d98fc4af77371e78b
I20260812 05:27:53.479848 6920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:27:53.503922 6920 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:27:53.504604 6920 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:27:53.510433 6920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a33ae696ada4f748d928ec807971d48: Generated new TSK 0
I20260812 05:27:53.510687 6920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:27:53.521230 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 6833
2026-08-12T05:27:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1431 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:27:53 dist-test-slave-zpfg krb5kdc[6937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:27:53 dist-test-slave-zpfg krb5kdc[6937](info): set up 2 sockets
Aug 12 05:27:53 dist-test-slave-zpfg krb5kdc[6937](info): commencing operation
krb5kdc: starting...
W20260812 05:27:54.627686 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:27:54 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:27:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:27:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:27:54.766482 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35429
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:35429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:54.860040 6952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:54.860296 6952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:54.860322 6952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:54.861725 6952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:27:54.861788 6952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:54.861804 6952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:54.861819 6952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:27:54.861833 6952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:27:54.863565 6952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:35429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:54.863807 6952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:54.864014 6952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:54.869656 6961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:54.869838 6952 server_base.cc:1034] running on GCE node
W20260812 05:27:54.869652 6959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:54.869697 6958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:54.870242 6952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:54.870579 6952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:54.871740 6952 hybrid_clock.cc:648] HybridClock initialized: now 1786512474871726 us; error 33 us; skew 500 ppm
Aug 12 05:27:54 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512474, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:54.879252 6952 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:27:54.879812 6952 webserver.cc:460] Webserver started at http://127.24.66.126:36095/ using document root <none> and password file <none>
I20260812 05:27:54.880086 6952 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:54.880163 6952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:54.880314 6952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:27:54.881448 6952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "10b8f42a14fd42718a018acb22a3bb3a"
format_stamp: "Formatted at 2026-08-12 05:27:54 on dist-test-slave-zpfg"
server_key: "17e4d4b4eac7bba6699fc29810aca005"
server_key_iv: "4ba5a8aaa3e76e3b66e38a04c27a70da"
server_key_version: "encryptionkey@0"
I20260812 05:27:54.881858 6952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "10b8f42a14fd42718a018acb22a3bb3a"
format_stamp: "Formatted at 2026-08-12 05:27:54 on dist-test-slave-zpfg"
server_key: "17e4d4b4eac7bba6699fc29810aca005"
server_key_iv: "4ba5a8aaa3e76e3b66e38a04c27a70da"
server_key_version: "encryptionkey@0"
I20260812 05:27:54.883428 6952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:27:54.884269 6968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:54.884469 6952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:54.884577 6952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "10b8f42a14fd42718a018acb22a3bb3a"
format_stamp: "Formatted at 2026-08-12 05:27:54 on dist-test-slave-zpfg"
server_key: "17e4d4b4eac7bba6699fc29810aca005"
server_key_iv: "4ba5a8aaa3e76e3b66e38a04c27a70da"
server_key_version: "encryptionkey@0"
I20260812 05:27:54.884656 6952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:54.900061 6952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:54.902724 6952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:54.902904 6952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:54.907557 6952 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:35429
I20260812 05:27:54.907601 7026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:35429 every 8 connection(s)
I20260812 05:27:54.907984 6952 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:27:54.909128 7027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:54.911600 7027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Bootstrap starting.
I20260812 05:27:54.912250 7027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:54.912566 7027 log.cc:824] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:54.913154 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 6952
I20260812 05:27:54.913282 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:27:54.913456 7027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: No bootstrap required, opened a new log
I20260812 05:27:54.913472 24841 external_mini_cluster.cc:1428] Setting key 3dcefe9ec0ed918c43b5e8b23a868a2f
I20260812 05:27:54.916131 7027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:54.916302 7027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:54.916330 7027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10b8f42a14fd42718a018acb22a3bb3a, State: Initialized, Role: FOLLOWER
I20260812 05:27:54.916499 7027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:54.916582 7027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:27:54.916635 7027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:27:54.916693 7027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:54.917452 7027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:54.917551 7027 leader_election.cc:302] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10b8f42a14fd42718a018acb22a3bb3a; no voters:
I20260812 05:27:54.917785 7027 leader_election.cc:288] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:27:54.917862 7032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:27:54.918056 7032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 1 LEADER]: Becoming Leader. State: Replica: 10b8f42a14fd42718a018acb22a3bb3a, State: Running, Role: LEADER
I20260812 05:27:54.918131 7027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:27:54.918221 7032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:54.918679 7032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10b8f42a14fd42718a018acb22a3bb3a. Latest consensus state: current_term: 1 leader_uuid: "10b8f42a14fd42718a018acb22a3bb3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } } }
I20260812 05:27:54.918694 7034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10b8f42a14fd42718a018acb22a3bb3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } } }
I20260812 05:27:54.918848 7032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:54.918856 7034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:54.919117 7042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:27:54.919850 7042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:27:54.921443 7042 catalog_manager.cc:1347] Generated new cluster ID: 8f71ab0720564675baac048dc0937cf7
I20260812 05:27:54.921509 7042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:27:54 dist-test-slave-zpfg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:54.927735 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:54.914086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42806 (local address 127.24.66.126:35429)
0812 05:27:54.914327 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:54.914373 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:54.914415 (+ 42us) server_negotiation.cc:407] Connection header received
0812 05:27:54.915242 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:54.915250 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:54.915306 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:54.915481 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:54.916157 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:54.917599 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:54.919003 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:54.919817 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:54.919922 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:54.923400 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:54.923404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:54.923405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:54.923418 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:54.925659 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:54.926116 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:54.926120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:54.926120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:54.926273 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:54.926524 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:54.926525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:54.926525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:54.926888 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:54.927066 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:54.927175 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 05:27:54.927493 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":98,"threads_started":1}
I20260812 05:27:54.944738 7042 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:27:54.945448 7042 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:27:54.950119 7042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Generated new TSK 0
I20260812 05:27:54.950384 7042 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:27:54.989775 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35429
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:55.097545 7054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:55.097783 7054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:55.097810 7054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:55.099013 7054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:55.099061 7054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:55.099087 7054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:27:55.100570 7054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:55.100772 7054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:55.100980 7054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:55.106542 7062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:55.106563 7060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:55.106611 7054 server_base.cc:1034] running on GCE node
W20260812 05:27:55.106734 7059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:55.106995 7054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:55.107256 7054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:55.108438 7054 hybrid_clock.cc:648] HybridClock initialized: now 1786512475108431 us; error 38 us; skew 500 ppm
Aug 12 05:27:55 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512475, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:55.111202 7054 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:27:55.111670 7054 webserver.cc:460] Webserver started at http://127.24.66.65:42229/ using document root <none> and password file <none>
I20260812 05:27:55.111929 7054 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:55.112002 7054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:55.112154 7054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:27:55.113251 7054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "dd65a69aa01043c3a74e4b17f6deee4c"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "b9d3af8120ade76fdf584d1e510cf57d"
server_key_iv: "4203a831127542bbf83b11c1b7a7e809"
server_key_version: "encryptionkey@0"
I20260812 05:27:55.113617 7054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "dd65a69aa01043c3a74e4b17f6deee4c"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "b9d3af8120ade76fdf584d1e510cf57d"
server_key_iv: "4203a831127542bbf83b11c1b7a7e809"
server_key_version: "encryptionkey@0"
I20260812 05:27:55.115147 7054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260812 05:27:55.116016 7069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:55.116262 7054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:55.116366 7054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "dd65a69aa01043c3a74e4b17f6deee4c"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "b9d3af8120ade76fdf584d1e510cf57d"
server_key_iv: "4203a831127542bbf83b11c1b7a7e809"
server_key_version: "encryptionkey@0"
I20260812 05:27:55.116467 7054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:55.144912 7054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:55.147090 7054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:55.147267 7054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:55.147563 7054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:55.147926 7054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:55.147987 7054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:55.148029 7054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:55.148057 7054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:55.154269 7054 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:37887
I20260812 05:27:55.154402 7191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:37887 every 8 connection(s)
I20260812 05:27:55.154731 7054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:27:55.157351 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 7054
I20260812 05:27:55.157460 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:27:55.157680 24841 external_mini_cluster.cc:1428] Setting key 93f985ab0a87cd45f57267347b26df57
Aug 12 05:27:55 dist-test-slave-zpfg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512475, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:55.165426 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.155671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:43221 (local address 127.24.66.126:35429)
0812 05:27:55.155745 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:55.155747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:55.155863 (+ 116us) server_negotiation.cc:407] Connection header received
0812 05:27:55.156609 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:55.156610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:55.156623 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:55.156682 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:55.157809 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.158428 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.159504 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.159825 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.159902 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.162407 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:55.162408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:55.162408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:55.162416 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:55.163924 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:55.164371 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:55.164372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:55.164373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:55.164434 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:55.164702 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:55.164703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:55.164703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:55.164826 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:55.164916 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:55.165206 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 05:27:55.165325 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 05:27:55.165566 7194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.155563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:43221)
0812 05:27:55.155731 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:55.155776 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:55.156290 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:55.156750 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:55.156753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:55.157391 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:55.157676 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.157682 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.158527 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.158528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.159408 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.159410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.159920 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.159921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.160032 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.160608 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:55.160616 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:55.162233 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:55.164029 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:55.164037 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:55.164040 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:55.164275 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:55.164511 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:55.164512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:55.164512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:55.164625 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:55.164910 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:55.164918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:55.165062 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 05:27:55.165170 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 05:27:55.165333 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260812 05:27:55.166060 7192 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35429
I20260812 05:27:55.166198 7192 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:55.166448 7192 heartbeater.cc:511] Master 127.24.66.126:35429 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
I20260812 05:27:55.167032 6987 ts_manager.cc:194] Registered new tserver with Master: dd65a69aa01043c3a74e4b17f6deee4c (127.24.66.65:37887)
I20260812 05:27:55.167665 6987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:43221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:27:55.189141 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35429
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:55.284713 7197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:55.284965 7197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:55.284993 7197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:55.286391 7197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:55.286458 7197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:55.286487 7197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:27:55.288157 7197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:55.288430 7197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:55.288651 7197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:55.294039 7203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:55.294030 7202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:55.294029 7205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:55.294534 7197 server_base.cc:1034] running on GCE node
I20260812 05:27:55.294776 7197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:55.295053 7197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:55.296273 7197 hybrid_clock.cc:648] HybridClock initialized: now 1786512475296248 us; error 42 us; skew 500 ppm
Aug 12 05:27:55 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512475, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:55.299041 7197 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:27:55.299546 7197 webserver.cc:460] Webserver started at http://127.24.66.66:40027/ using document root <none> and password file <none>
I20260812 05:27:55.299741 7197 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:55.299782 7197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:55.299875 7197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:27:55.300944 7197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "ddbb3333d8b74a9e89463bf4663ffeaa"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "9ab6d0fcf8a5422e7970944193b0ac95"
server_key_iv: "f12717edcfff4c67b695dab49b65cc16"
server_key_version: "encryptionkey@0"
I20260812 05:27:55.301319 7197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "ddbb3333d8b74a9e89463bf4663ffeaa"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "9ab6d0fcf8a5422e7970944193b0ac95"
server_key_iv: "f12717edcfff4c67b695dab49b65cc16"
server_key_version: "encryptionkey@0"
I20260812 05:27:55.302737 7197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:27:55.303527 7212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:55.303767 7197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:27:55.303848 7197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "ddbb3333d8b74a9e89463bf4663ffeaa"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "9ab6d0fcf8a5422e7970944193b0ac95"
server_key_iv: "f12717edcfff4c67b695dab49b65cc16"
server_key_version: "encryptionkey@0"
I20260812 05:27:55.303900 7197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:55.316402 7197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:55.318796 7197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:55.318959 7197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:55.319288 7197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:55.319703 7197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:55.319742 7197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:55.319766 7197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:55.319778 7197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:55.326399 7197 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:36507
I20260812 05:27:55.326428 7329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:36507 every 8 connection(s)
I20260812 05:27:55.326777 7197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:27:55 dist-test-slave-zpfg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512475, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:55.335544 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 7197
I20260812 05:27:55.335645 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:27:55.335891 24841 external_mini_cluster.cc:1428] Setting key b09cfad6d28f6804535abe6bb99a86bf
I20260812 05:27:55.337028 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.327550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:40253 (local address 127.24.66.126:35429)
0812 05:27:55.327625 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:55.327627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:55.327901 (+ 274us) server_negotiation.cc:407] Connection header received
0812 05:27:55.328433 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:55.328434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:55.328448 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:55.328507 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:55.329581 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.330107 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.331243 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.331447 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.331539 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.333782 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:55.333784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:55.333784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:55.333793 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:55.335241 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:55.335742 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:55.335748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:55.335748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:55.335800 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:55.336107 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:55.336108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:55.336108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:55.336261 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:55.336334 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:55.336734 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 05:27:55.336905 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:27:55.337148 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.327537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:40253)
0812 05:27:55.327717 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:55.327756 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:55.328305 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:55.328584 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:55.328588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:55.329216 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:55.329467 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.329472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.330218 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.330219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.331136 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.331138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.331552 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.331553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.331687 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.332203 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:55.332212 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:55.333677 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:55.335356 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:55.335364 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:55.335366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:55.335627 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:55.335877 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:55.335878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:55.335879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:55.336022 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:55.336372 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:55.336381 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:55.336560 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 05:27:55.336715 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 05:27:55.336921 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260812 05:27:55.338773 7330 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35429
I20260812 05:27:55.338912 7330 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:55.339116 7330 heartbeater.cc:511] Master 127.24.66.126:35429 requested a full tablet report, sending...
I20260812 05:27:55.339572 6987 ts_manager.cc:194] Registered new tserver with Master: ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507)
I20260812 05:27:55.340046 6987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:40253
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:27:55.365892 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35429
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:55.460347 7336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:55.460613 7336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:55.460659 7336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:55.461998 7336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:55.462078 7336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:55.462131 7336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:27:55.463929 7336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:55.464257 7336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:55.464540 7336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:55.470261 7342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:55.470273 7344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:55.470327 7341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:55.470527 7336 server_base.cc:1034] running on GCE node
I20260812 05:27:55.470748 7336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:55.471098 7336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:55.472278 7336 hybrid_clock.cc:648] HybridClock initialized: now 1786512475472272 us; error 38 us; skew 500 ppm
Aug 12 05:27:55 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512475, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:55.475172 7336 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:27:55.475703 7336 webserver.cc:460] Webserver started at http://127.24.66.67:44635/ using document root <none> and password file <none>
I20260812 05:27:55.475996 7336 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:55.476078 7336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:55.476272 7336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:27:55.477456 7336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "9c96585a2d0343c1a89e939f003aae2a"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "e6d8e43ce12c61349d3278ed48ad8a9c"
server_key_iv: "627eea687e77352c293a3972502d356c"
server_key_version: "encryptionkey@0"
I20260812 05:27:55.477880 7336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "9c96585a2d0343c1a89e939f003aae2a"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "e6d8e43ce12c61349d3278ed48ad8a9c"
server_key_iv: "627eea687e77352c293a3972502d356c"
server_key_version: "encryptionkey@0"
I20260812 05:27:55.479542 7336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 05:27:55.480485 7352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:55.480723 7336 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:27:55.480858 7336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "9c96585a2d0343c1a89e939f003aae2a"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "e6d8e43ce12c61349d3278ed48ad8a9c"
server_key_iv: "627eea687e77352c293a3972502d356c"
server_key_version: "encryptionkey@0"
I20260812 05:27:55.480939 7336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:55.497103 7336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:55.499611 7336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:55.499825 7336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:55.500104 7336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:55.500478 7336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:55.500538 7336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:55.500583 7336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:55.500608 7336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:55.507094 7336 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:43703
I20260812 05:27:55.507140 7472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:43703 every 8 connection(s)
I20260812 05:27:55.507483 7336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:27:55.512691 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 7336
I20260812 05:27:55.512806 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:27:55.513008 24841 external_mini_cluster.cc:1428] Setting key ccf2ce16cb064b1eb71852c76287a0b6
Aug 12 05:27:55 dist-test-slave-zpfg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512475, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:55.517289 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.508357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:38593 (local address 127.24.66.126:35429)
0812 05:27:55.508424 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:55.508426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:55.508647 (+ 221us) server_negotiation.cc:407] Connection header received
0812 05:27:55.509202 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:55.509203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:55.509213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:55.509278 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:55.510243 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.510851 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.511891 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.512080 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.512135 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.514730 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:55.514731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:55.514731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:55.514738 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:55.516055 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:55.516453 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:55.516454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:55.516455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:55.516507 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:55.516724 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:55.516725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:55.516725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:55.516855 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:55.516908 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:55.517154 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 05:27:55.517221 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:27:55.517489 7475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.508330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:38593)
0812 05:27:55.508506 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:55.508550 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:55.509094 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:55.509334 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:55.509338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:55.509945 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:55.510151 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.510155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.510946 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.510946 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.511809 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.511812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.512192 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.512193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.512340 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.512976 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:55.512987 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:55.514645 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:55.516140 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:55.516147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:55.516149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:55.516386 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:55.516567 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:55.516568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:55.516568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:55.516671 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:55.516912 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:55.516916 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:55.517039 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 05:27:55.517124 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 05:27:55.517254 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260812 05:27:55.518158 7473 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35429
I20260812 05:27:55.518304 7473 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:55.518532 7473 heartbeater.cc:511] Master 127.24.66.126:35429 requested a full tablet report, sending...
I20260812 05:27:55.519078 6987 ts_manager.cc:194] Registered new tserver with Master: 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703)
I20260812 05:27:55.519562 6987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:38593
I20260812 05:27:55.524915 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:27:55.533353 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.526373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42812 (local address 127.24.66.126:35429)
0812 05:27:55.526424 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:55.526426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:55.526511 (+ 85us) server_negotiation.cc:407] Connection header received
0812 05:27:55.526571 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:55.526572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:55.526581 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:55.526635 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:55.527269 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.527864 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.528623 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.528817 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.528879 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.530244 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:55.530245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:55.530245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:55.530251 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:55.531891 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:55.532234 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:55.532235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:55.532235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:55.532291 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:55.532642 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:55.532642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:55.532643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:55.532827 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:55.532948 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:55.533062 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 05:27:55.533138 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:27:55.535328 6987 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42812:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:27:55.537025 6987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:27:55.543897 7492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.539659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.24.66.66:36507)
0812 05:27:55.539862 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:55.539866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:55.539877 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:27:55.539916 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:55.539918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:55.539934 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:55.540017 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:55.540370 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.541094 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.542915 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.543481 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.543570 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.543634 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:55.543759 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 05:27:55.543801 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":88,"threads_started":1}
I20260812 05:27:55.543901 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.539539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:36507 (local address 127.0.0.1:33922)
0812 05:27:55.539714 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:55.539723 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:55.539773 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:55.540040 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:55.540043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:55.540046 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:55.540149 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.540152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.541440 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.541441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.542855 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.542857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.543570 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.543571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.543650 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.543664 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 05:27:55.543742 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 05:27:55.543809 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":102,"thread_start_us":78,"threads_started":1}
I20260812 05:27:55.544005 7490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.539422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.24.66.67:43703)
0812 05:27:55.539650 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:55.539655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:55.539743 (+ 88us) server_negotiation.cc:407] Connection header received
0812 05:27:55.539889 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:55.539893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:55.539914 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:55.539958 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:55.540249 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.541431 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.542698 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.543508 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.543587 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.543651 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:55.543879 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 05:27:55.543929 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":90,"threads_started":1}
I20260812 05:27:55.544158 7487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.539283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:37887 (local address 127.0.0.1:55810)
0812 05:27:55.539623 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:55.539667 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:55.539768 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:55.540133 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:55.540134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:55.540136 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:55.540196 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.540197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.541524 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.541524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.542775 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.542777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.543704 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.543705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.543769 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.543772 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:27:55.543818 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 05:27:55.543869 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":34,"thread_start_us":98,"threads_started":1}
I20260812 05:27:55.544248 7491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.539285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:43703 (local address 127.0.0.1:40994)
0812 05:27:55.539668 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:55.539678 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:55.539773 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:55.540076 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:55.540076 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:55.540078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:55.540149 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.540150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.541552 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.541553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.542609 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.542611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.543692 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.543693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.543761 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.543766 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:27:55.544137 (+ 371us) client_negotiation.cc:240] Negotiation successful
0812 05:27:55.544191 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":102,"threads_started":1}
I20260812 05:27:55.544762 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.539377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55810 (local address 127.24.66.65:37887)
0812 05:27:55.539555 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:55.539560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:55.539868 (+ 308us) server_negotiation.cc:407] Connection header received
0812 05:27:55.539924 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:55.539926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:55.539941 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:55.540017 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:55.540271 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.541424 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.542866 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.543568 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.543626 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.543675 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:55.543946 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 05:27:55.544701 (+ 755us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260812 05:27:55.544715 7261 tablet_service.cc:1467] Processing CreateTablet for tablet a47f49e9562c41f6b65310ad94fbe4e3 (DEFAULT_TABLE table=test-table [id=ff6b5700681049ff9e7e88caeb00abd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:27:55.545104 7261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a47f49e9562c41f6b65310ad94fbe4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:55.547808 7119 tablet_service.cc:1467] Processing CreateTablet for tablet a47f49e9562c41f6b65310ad94fbe4e3 (DEFAULT_TABLE table=test-table [id=ff6b5700681049ff9e7e88caeb00abd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:27:55.547832 7402 tablet_service.cc:1467] Processing CreateTablet for tablet a47f49e9562c41f6b65310ad94fbe4e3 (DEFAULT_TABLE table=test-table [id=ff6b5700681049ff9e7e88caeb00abd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:27:55.548017 7493 tablet_bootstrap.cc:492] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: Bootstrap starting.
I20260812 05:27:55.548187 7119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a47f49e9562c41f6b65310ad94fbe4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:55.548190 7402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a47f49e9562c41f6b65310ad94fbe4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:55.548695 7493 tablet_bootstrap.cc:654] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:55.549305 7493 log.cc:824] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:55.550207 7493 tablet_bootstrap.cc:492] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: No bootstrap required, opened a new log
I20260812 05:27:55.550292 7493 ts_tablet_manager.cc:1397] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:27:55.551153 7495 tablet_bootstrap.cc:492] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: Bootstrap starting.
I20260812 05:27:55.551717 7495 tablet_bootstrap.cc:654] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:55.551990 7496 tablet_bootstrap.cc:492] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Bootstrap starting.
I20260812 05:27:55.552390 7495 log.cc:824] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:55.552685 7496 tablet_bootstrap.cc:654] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:55.552551 7493 raft_consensus.cc:348] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } }
I20260812 05:27:55.552757 7493 raft_consensus.cc:374] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:55.552811 7493 raft_consensus.cc:729] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddbb3333d8b74a9e89463bf4663ffeaa, State: Initialized, Role: FOLLOWER
I20260812 05:27:55.552959 7493 consensus_queue.cc:260] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } }
I20260812 05:27:55.553200 7495 tablet_bootstrap.cc:492] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: No bootstrap required, opened a new log
I20260812 05:27:55.553257 7493 ts_tablet_manager.cc:1428] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 05:27:55.553275 7495 ts_tablet_manager.cc:1397] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:27:55.553364 7496 log.cc:824] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:55.553390 7330 heartbeater.cc:503] Master 127.24.66.126:35429 was elected leader, sending a full tablet report...
I20260812 05:27:55.554167 7496 tablet_bootstrap.cc:492] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: No bootstrap required, opened a new log
I20260812 05:27:55.554322 7496 ts_tablet_manager.cc:1397] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:27:55.555423 7495 raft_consensus.cc:348] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } }
I20260812 05:27:55.555569 7495 raft_consensus.cc:374] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:55.555593 7495 raft_consensus.cc:729] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c96585a2d0343c1a89e939f003aae2a, State: Initialized, Role: FOLLOWER
I20260812 05:27:55.555720 7495 consensus_queue.cc:260] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } }
I20260812 05:27:55.556033 7495 ts_tablet_manager.cc:1428] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:27:55.556054 7473 heartbeater.cc:503] Master 127.24.66.126:35429 was elected leader, sending a full tablet report...
I20260812 05:27:55.556361 7496 raft_consensus.cc:348] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } }
I20260812 05:27:55.556521 7496 raft_consensus.cc:374] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:55.556556 7496 raft_consensus.cc:729] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd65a69aa01043c3a74e4b17f6deee4c, State: Initialized, Role: FOLLOWER
I20260812 05:27:55.556659 7496 consensus_queue.cc:260] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } }
I20260812 05:27:55.556957 7496 ts_tablet_manager.cc:1428] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 05:27:55.556978 7192 heartbeater.cc:503] Master 127.24.66.126:35429 was elected leader, sending a full tablet report...
W20260812 05:27:55.577340 7332 tablet.cc:2369] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:27:55.655606 7193 tablet.cc:2369] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:27:55.758090 7474 tablet.cc:2369] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:27:55.760861 7498 raft_consensus.cc:482] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:27:55.760984 7498 raft_consensus.cc:504] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } }
I20260812 05:27:55.761333 7498 leader_election.cc:288] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd65a69aa01043c3a74e4b17f6deee4c (127.24.66.65:37887), 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703)
I20260812 05:27:55.764760 7503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.761550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:37887 (local address 127.24.66.66:35653)
0812 05:27:55.761905 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:55.761916 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:55.761976 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:55.762123 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:55.762124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:55.762126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:55.762196 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.762198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.763121 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.763122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.763874 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.763876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.764485 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.764486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.764562 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.764567 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:27:55.764620 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 05:27:55.764652 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":133,"thread_start_us":113,"threads_started":1}
I20260812 05:27:55.764811 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.761790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:35653 (local address 127.24.66.65:37887)
0812 05:27:55.761843 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:55.761845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:55.761984 (+ 139us) server_negotiation.cc:407] Connection header received
0812 05:27:55.762025 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:55.762026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:55.762039 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:55.762103 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:55.762317 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.763008 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.763991 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.764390 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.764452 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.764460 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:55.764657 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 05:27:55.764712 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 05:27:55.764854 7490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.761652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:54125 (local address 127.24.66.67:43703)
0812 05:27:55.761752 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:55.761754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:55.761765 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:27:55.761831 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:55.761832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:55.761845 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:55.761924 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:55.762124 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.762803 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.764012 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.764410 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.764509 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.764518 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:55.764739 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 05:27:55.764783 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 05:27:55.764902 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.761550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:43703 (local address 127.24.66.66:54125)
0812 05:27:55.761610 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:55.761625 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:55.761680 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:55.761936 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:55.761936 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:55.761938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:55.762038 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.762040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.762895 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.762896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.763874 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:55.763877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.764533 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.764535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:55.764653 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.764657 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:27:55.764719 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 05:27:55.764761 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 05:27:55.765429 7141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a47f49e9562c41f6b65310ad94fbe4e3" candidate_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" is_pre_election: true
I20260812 05:27:55.765622 7141 raft_consensus.cc:2454] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddbb3333d8b74a9e89463bf4663ffeaa in term 0.
I20260812 05:27:55.765550 7424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a47f49e9562c41f6b65310ad94fbe4e3" candidate_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c96585a2d0343c1a89e939f003aae2a" is_pre_election: true
I20260812 05:27:55.765741 7424 raft_consensus.cc:2454] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddbb3333d8b74a9e89463bf4663ffeaa in term 0.
I20260812 05:27:55.765887 7216 leader_election.cc:302] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd65a69aa01043c3a74e4b17f6deee4c, ddbb3333d8b74a9e89463bf4663ffeaa; no voters:
I20260812 05:27:55.766014 7498 raft_consensus.cc:2781] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:27:55.766065 7498 raft_consensus.cc:482] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:27:55.766093 7498 raft_consensus.cc:3037] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:55.767000 7498 raft_consensus.cc:504] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } }
I20260812 05:27:55.767153 7498 leader_election.cc:288] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [CANDIDATE]: Term 1 election: Requested vote from peers dd65a69aa01043c3a74e4b17f6deee4c (127.24.66.65:37887), 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703)
I20260812 05:27:55.767287 7141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a47f49e9562c41f6b65310ad94fbe4e3" candidate_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd65a69aa01043c3a74e4b17f6deee4c"
I20260812 05:27:55.767328 7424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a47f49e9562c41f6b65310ad94fbe4e3" candidate_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c96585a2d0343c1a89e939f003aae2a"
I20260812 05:27:55.767392 7141 raft_consensus.cc:3037] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:55.767427 7424 raft_consensus.cc:3037] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:55.768131 7141 raft_consensus.cc:2454] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddbb3333d8b74a9e89463bf4663ffeaa in term 1.
I20260812 05:27:55.768278 7424 raft_consensus.cc:2454] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ddbb3333d8b74a9e89463bf4663ffeaa in term 1.
I20260812 05:27:55.768389 7216 leader_election.cc:302] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd65a69aa01043c3a74e4b17f6deee4c, ddbb3333d8b74a9e89463bf4663ffeaa; no voters:
I20260812 05:27:55.768503 7498 raft_consensus.cc:2781] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:27:55.768709 7498 raft_consensus.cc:686] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 1 LEADER]: Becoming Leader. State: Replica: ddbb3333d8b74a9e89463bf4663ffeaa, State: Running, Role: LEADER
I20260812 05:27:55.768842 7498 consensus_queue.cc:237] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } }
I20260812 05:27:55.769796 6987 catalog_manager.cc:5495] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa reported cstate change: term changed from 0 to 1, leader changed from <none> to ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66). New cstate: current_term: 1 leader_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } health_report { overall_health: HEALTHY } } }
I20260812 05:27:55.828226 7492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:55.824868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33934 (local address 127.24.66.66:36507)
0812 05:27:55.824932 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:55.824934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:55.824945 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:27:55.824999 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:55.825000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:55.825029 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:55.825109 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:27:55.825289 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.826012 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.826885 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:55.827234 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:55.827306 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:55.827590 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:27:55.827708 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:27:55.827785 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:55.827897 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 05:27:55.827920 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:27:55.830178 7141 raft_consensus.cc:1261] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Refusing update from remote peer ddbb3333d8b74a9e89463bf4663ffeaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:27:55.830176 7424 raft_consensus.cc:1261] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a [term 1 FOLLOWER]: Refusing update from remote peer ddbb3333d8b74a9e89463bf4663ffeaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:27:55.830646 7498 consensus_queue.cc:1035] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:27:55.830807 7504 consensus_queue.cc:1035] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:27:55.833540 7508 mvcc.cc:205] Tried to move back new op lower bound from 7317555100997464064 to 7317555100749975552. Current Snapshot: MvccSnapshot[applied={T|T < 7317555100997464064 or (T in {7317555100997464064})}]
I20260812 05:27:55.835119 7510 mvcc.cc:205] Tried to move back new op lower bound from 7317555100997464064 to 7317555100749975552. Current Snapshot: MvccSnapshot[applied={T|T < 7317555100997464064 or (T in {7317555100997464064})}]
I20260812 05:27:55.836499 6987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:27:55.836604 6987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:27:55.837363 6987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Sending DeleteTablet for 3 replicas of tablet a47f49e9562c41f6b65310ad94fbe4e3
I20260812 05:27:55.837713 7119 tablet_service.cc:1514] Processing DeleteTablet for tablet a47f49e9562c41f6b65310ad94fbe4e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:27:55 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:55810
I20260812 05:27:55.837713 7261 tablet_service.cc:1514] Processing DeleteTablet for tablet a47f49e9562c41f6b65310ad94fbe4e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:27:55 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:33922
I20260812 05:27:55.838001 7518 tablet_replica.cc:330] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: stopping tablet replica
I20260812 05:27:55.837999 7402 tablet_service.cc:1514] Processing DeleteTablet for tablet a47f49e9562c41f6b65310ad94fbe4e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:27:55 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:40994
I20260812 05:27:55.838129 7518 raft_consensus.cc:2229] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:27:55.838279 7519 tablet_replica.cc:330] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: stopping tablet replica
I20260812 05:27:55.838413 7518 raft_consensus.cc:2258] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:27:55.838420 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 7054
I20260812 05:27:55.838435 7519 raft_consensus.cc:2229] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:27:55.838538 7519 raft_consensus.cc:2258] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:27:55.843752 7517 tablet_replica.cc:330] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: stopping tablet replica
I20260812 05:27:55.843910 7517 raft_consensus.cc:2229] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:27:55.844022 7517 raft_consensus.cc:2258] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:27:55.844498 7517 ts_tablet_manager.cc:1905] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:27:55.846196 7517 ts_tablet_manager.cc:1918] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:27:55.846294 7517 log.cc:1196] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/a47f49e9562c41f6b65310ad94fbe4e3
I20260812 05:27:55.846702 7517 ts_tablet_manager.cc:1939] T a47f49e9562c41f6b65310ad94fbe4e3 P ddbb3333d8b74a9e89463bf4663ffeaa: Deleting consensus metadata
I20260812 05:27:55.847203 6972 catalog_manager.cc:4841] TS ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507): tablet a47f49e9562c41f6b65310ad94fbe4e3 (table test-table [id=ff6b5700681049ff9e7e88caeb00abd5]) successfully deleted
W20260812 05:27:55.848436 6972 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:37887 (error 108)
I20260812 05:27:55.849092 7519 ts_tablet_manager.cc:1905] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 05:27:55.850587 7491 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:27:55.848858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:37887 (local address 127.0.0.1:55816)
0812 05:27:55.848917 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:55.848929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:55.848977 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:55.850523 (+ 1546us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:37887: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 05:27:55.850790 7519 ts_tablet_manager.cc:1918] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260812 05:27:55.850835 6972 catalog_manager.cc:4584] TS dd65a69aa01043c3a74e4b17f6deee4c (127.24.66.65:37887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a47f49e9562c41f6b65310ad94fbe4e3: Network error: Client connection negotiation failed: client connection to 127.24.66.65:37887: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 05:27:55.850898 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 7197
I20260812 05:27:55.851965 7519 log.cc:1196] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/a47f49e9562c41f6b65310ad94fbe4e3
I20260812 05:27:55.852351 7519 ts_tablet_manager.cc:1939] T a47f49e9562c41f6b65310ad94fbe4e3 P 9c96585a2d0343c1a89e939f003aae2a: Deleting consensus metadata
I20260812 05:27:55.852838 6971 catalog_manager.cc:4841] TS 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703): tablet a47f49e9562c41f6b65310ad94fbe4e3 (table test-table [id=ff6b5700681049ff9e7e88caeb00abd5]) successfully deleted
I20260812 05:27:55.858507 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 7336
I20260812 05:27:55.866011 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 6952
I20260812 05:27:55.873055 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35429
--webserver_interface=127.24.66.126
--webserver_port=36095
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:35429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:55.960425 7521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:55.960665 7521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:55.960690 7521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:55.962054 7521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:27:55.962110 7521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:55.962126 7521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:55.962138 7521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:27:55.962150 7521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:27:55.963692 7521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:35429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=36095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:55.963907 7521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:55.964100 7521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:55.974867 7531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:55.974876 7529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:55.974866 7528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:55.975140 7521 server_base.cc:1034] running on GCE node
I20260812 05:27:55.975659 7521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:55.975971 7521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:55.977246 7521 hybrid_clock.cc:648] HybridClock initialized: now 1786512475977170 us; error 86 us; skew 500 ppm
Aug 12 05:27:55 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512475, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:55.980422 7521 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:27:55.980931 7521 webserver.cc:460] Webserver started at http://127.24.66.126:36095/ using document root <none> and password file <none>
I20260812 05:27:55.981195 7521 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:55.981259 7521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:55.982921 7521 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:27:55.983685 7538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:55.983863 7521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:27:55.983966 7521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "10b8f42a14fd42718a018acb22a3bb3a"
format_stamp: "Formatted at 2026-08-12 05:27:54 on dist-test-slave-zpfg"
server_key: "17e4d4b4eac7bba6699fc29810aca005"
server_key_iv: "4ba5a8aaa3e76e3b66e38a04c27a70da"
server_key_version: "encryptionkey@0"
I20260812 05:27:55.984365 7521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:56.013178 7521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:56.016084 7521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:56.016239 7521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:56.020937 7521 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:35429
I20260812 05:27:56.021040 7593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:35429 every 8 connection(s)
I20260812 05:27:56.021442 7521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:27:56.023489 7594 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:27:56.024231 7594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Bootstrap starting.
I20260812 05:27:56.029780 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 7521
I20260812 05:27:56.030115 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:37887
--local_ip_for_outbound_sockets=127.24.66.65
--tserver_master_addrs=127.24.66.126:35429
--webserver_port=42229
--webserver_interface=127.24.66.65
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:27:56.034469 7594 log.cc:824] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:56.036623 7594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:27:56.036965 7594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Bootstrap complete.
I20260812 05:27:56.039530 7594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:56.039796 7594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10b8f42a14fd42718a018acb22a3bb3a, State: Initialized, Role: FOLLOWER
I20260812 05:27:56.039996 7594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:56.040079 7594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:27:56.040170 7594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:27:56.040227 7594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:27:56.041309 7594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:56.041445 7594 leader_election.cc:302] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10b8f42a14fd42718a018acb22a3bb3a; no voters:
I20260812 05:27:56.041666 7594 leader_election.cc:288] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 05:27:56.041705 7600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 2 FOLLOWER]: Leader election won for term 2
I20260812 05:27:56.041828 7600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 2 LEADER]: Becoming Leader. State: Replica: 10b8f42a14fd42718a018acb22a3bb3a, State: Running, Role: LEADER
I20260812 05:27:56.041973 7594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:27:56.041987 7600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:56.042448 7601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "10b8f42a14fd42718a018acb22a3bb3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } } }
I20260812 05:27:56.042542 7601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:56.042968 7602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10b8f42a14fd42718a018acb22a3bb3a. Latest consensus state: current_term: 2 leader_uuid: "10b8f42a14fd42718a018acb22a3bb3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } } }
I20260812 05:27:56.043054 7602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:56.043172 7607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:27:56.044055 7607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:27:56.044219 7607 catalog_manager.cc:1259] Loaded cluster ID: 8f71ab0720564675baac048dc0937cf7
I20260812 05:27:56.044502 7607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:27:56.044782 7607 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 05:27:56.049476 7607 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 05:27:56.052304 7607 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:27:56.052546 7607 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Loaded TSK: 0
I20260812 05:27:56.052980 7607 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 05:27:56.118273 7596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:56.118588 7596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:56.118634 7596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:56.119868 7596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:56.119947 7596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:56.119994 7596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:27:56.121613 7596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:37887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=42229
--tserver_master_addrs=127.24.66.126:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:56.121920 7596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:56.122177 7596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:56.127877 7626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:56.128019 7624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:56.127887 7628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:56.129302 7596 server_base.cc:1034] running on GCE node
I20260812 05:27:56.129572 7596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:56.129861 7596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:56.131016 7596 hybrid_clock.cc:648] HybridClock initialized: now 1786512476130997 us; error 37 us; skew 500 ppm
Aug 12 05:27:56 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512476, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:56.133654 7596 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:27:56.134212 7596 webserver.cc:460] Webserver started at http://127.24.66.65:42229/ using document root <none> and password file <none>
I20260812 05:27:56.134500 7596 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:56.134562 7596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:56.136127 7596 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:56.136950 7635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:56.137128 7596 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:56.137300 7596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "dd65a69aa01043c3a74e4b17f6deee4c"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "b9d3af8120ade76fdf584d1e510cf57d"
server_key_iv: "4203a831127542bbf83b11c1b7a7e809"
server_key_version: "encryptionkey@0"
I20260812 05:27:56.137655 7596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:56.160622 7596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:56.163381 7596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:56.163549 7596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:56.163879 7596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:56.164460 7642 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 05:27:56.166057 7596 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 05:27:56.166112 7596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 05:27:56.166150 7596 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 05:27:56.167176 7596 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 05:27:56.167228 7596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 05:27:56.167282 7642 tablet_bootstrap.cc:492] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Bootstrap starting.
I20260812 05:27:56.175544 7596 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:37887
I20260812 05:27:56.175700 7752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:37887 every 8 connection(s)
I20260812 05:27:56.176133 7596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:27:56.176571 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 7596
I20260812 05:27:56.176964 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:36507
--local_ip_for_outbound_sockets=127.24.66.66
--tserver_master_addrs=127.24.66.126:35429
--webserver_port=40027
--webserver_interface=127.24.66.66
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:27:56.179689 7642 log.cc:824] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:56.182761 7642 tablet_bootstrap.cc:492] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:27:56.183122 7642 tablet_bootstrap.cc:492] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Bootstrap complete.
I20260812 05:27:56.184506 7642 ts_tablet_manager.cc:1397] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.005s
Aug 12 05:27:56 dist-test-slave-zpfg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512476, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:56.186414 7642 raft_consensus.cc:348] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } }
I20260812 05:27:56.186820 7642 raft_consensus.cc:729] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd65a69aa01043c3a74e4b17f6deee4c, State: Initialized, Role: FOLLOWER
I20260812 05:27:56.187001 7642 consensus_queue.cc:260] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } }
I20260812 05:27:56.187392 7642 ts_tablet_manager.cc:1428] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20260812 05:27:56.191272 7756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.179219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:59761)
0812 05:27:56.179992 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:56.180039 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:56.180636 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:56.181109 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:56.181113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:56.181620 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:56.181822 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.181828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.183144 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.183145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.184154 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.184157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.184855 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.184856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.184995 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.185576 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:56.185584 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:56.187279 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:56.189378 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:56.189386 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:56.189388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:56.189625 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:56.189921 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:56.189921 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:56.189922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:56.190038 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:56.190607 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:56.190614 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:56.190760 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 05:27:56.190869 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 05:27:56.191053 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":117,"threads_started":1}
I20260812 05:27:56.191272 7757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.179298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:59761 (local address 127.24.66.126:35429)
0812 05:27:56.179501 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:56.179513 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:56.180161 (+ 648us) server_negotiation.cc:407] Connection header received
0812 05:27:56.180882 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:56.180887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:56.180933 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:56.181080 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:56.181971 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.183041 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.184273 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.184737 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.184862 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.187455 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:56.187459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:56.187459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:56.187471 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:56.189254 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:56.189738 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:56.189742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:56.189742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:56.189845 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:56.190113 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:56.190114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:56.190115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:56.190519 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:56.190629 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:56.190936 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 05:27:56.191078 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":83,"threads_started":1}
I20260812 05:27:56.191987 7753 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35429
I20260812 05:27:56.192108 7753 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:56.192312 7753 heartbeater.cc:511] Master 127.24.66.126:35429 requested a full tablet report, sending...
I20260812 05:27:56.193060 7556 ts_manager.cc:194] Registered new tserver with Master: dd65a69aa01043c3a74e4b17f6deee4c (127.24.66.65:37887)
I20260812 05:27:56.194240 7556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:59761
I20260812 05:27:56.195163 7753 heartbeater.cc:503] Master 127.24.66.126:35429 was elected leader, sending a full tablet report...
Aug 12 05:27:56 dist-test-slave-zpfg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512475, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:27:56.203877 7760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.193688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:37887 (local address 127.0.0.1:55832)
0812 05:27:56.193903 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:56.193952 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:56.194028 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:56.194625 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:56.194631 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:56.194982 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:56.195058 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.195060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.196418 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.196419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.197062 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.197064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.197730 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.197731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.197791 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.198335 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:56.198407 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:56.200034 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:56.202508 (+ 2474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:56.202516 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:56.202518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:56.202752 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:56.203031 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:56.203032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:56.203032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:56.203105 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:56.203461 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:56.203469 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:56.203520 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 05:27:56.203590 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 05:27:56.203711 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":124,"threads_started":1}
I20260812 05:27:56.203879 7761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.193973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.24.66.65:37887)
0812 05:27:56.194440 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:56.194445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:56.194455 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:27:56.194489 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:56.194491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:56.194506 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:56.195151 (+ 645us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:56.195207 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.196324 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.197168 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.197641 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.197717 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.200139 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:56.200149 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:56.200149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:56.200158 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:56.202385 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:56.202847 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:56.202852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:56.202852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:56.202927 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:56.203184 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:56.203185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:56.203185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:56.203380 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:56.203482 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:56.203650 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 05:27:56.203783 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":76,"threads_started":1}
I20260812 05:27:56.204452 7684 tablet_service.cc:1514] Processing DeleteTablet for tablet a47f49e9562c41f6b65310ad94fbe4e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:27:55 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:55832
I20260812 05:27:56.205120 7762 tablet_replica.cc:330] stopping tablet replica
I20260812 05:27:56.205255 7762 raft_consensus.cc:2229] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:27:56.205364 7762 raft_consensus.cc:2258] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:27:56.205721 7762 ts_tablet_manager.cc:1905] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:27:56.208272 7762 ts_tablet_manager.cc:1918] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:27:56.208364 7762 log.cc:1196] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal/wals/a47f49e9562c41f6b65310ad94fbe4e3
I20260812 05:27:56.208642 7762 ts_tablet_manager.cc:1939] T a47f49e9562c41f6b65310ad94fbe4e3 P dd65a69aa01043c3a74e4b17f6deee4c: Deleting consensus metadata
I20260812 05:27:56.209221 7542 catalog_manager.cc:4841] TS dd65a69aa01043c3a74e4b17f6deee4c (127.24.66.65:37887): tablet a47f49e9562c41f6b65310ad94fbe4e3 (table test-table [id=ff6b5700681049ff9e7e88caeb00abd5]) successfully deleted
W20260812 05:27:56.270695 7755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:56.270917 7755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:56.270942 7755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:56.272151 7755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:56.272220 7755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:56.272287 7755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:27:56.273867 7755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:36507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=40027
--tserver_master_addrs=127.24.66.126:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:56.274080 7755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:56.274278 7755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:56.279920 7770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:56.279910 7768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:56.279917 7767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:56.279947 7755 server_base.cc:1034] running on GCE node
I20260812 05:27:56.280468 7755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:56.280742 7755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:56.281929 7755 hybrid_clock.cc:648] HybridClock initialized: now 1786512476281912 us; error 32 us; skew 500 ppm
Aug 12 05:27:56 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512476, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:56.285149 7755 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:27:56.285704 7755 webserver.cc:460] Webserver started at http://127.24.66.66:40027/ using document root <none> and password file <none>
I20260812 05:27:56.285974 7755 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:56.286073 7755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:56.287951 7755 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:56.288822 7778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:56.289088 7755 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:56.289212 7755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "ddbb3333d8b74a9e89463bf4663ffeaa"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "9ab6d0fcf8a5422e7970944193b0ac95"
server_key_iv: "f12717edcfff4c67b695dab49b65cc16"
server_key_version: "encryptionkey@0"
I20260812 05:27:56.289646 7755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:56.312088 7755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:56.314810 7755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:56.315006 7755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:56.315302 7755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:56.315707 7755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:56.315763 7755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:56.315799 7755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:56.315826 7755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:56.322062 7755 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:36507
I20260812 05:27:56.322113 7894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:36507 every 8 connection(s)
I20260812 05:27:56.322502 7755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:27:56.325843 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 7755
I20260812 05:27:56.326159 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:43703
--local_ip_for_outbound_sockets=127.24.66.67
--tserver_master_addrs=127.24.66.126:35429
--webserver_port=44635
--webserver_interface=127.24.66.67
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:27:56 dist-test-slave-zpfg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512476, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:56.334168 7757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.323551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:34357 (local address 127.24.66.126:35429)
0812 05:27:56.323626 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:56.323628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:56.323642 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:27:56.324444 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:56.324446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:56.324463 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:56.324532 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:56.325580 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.326228 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.327517 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.327870 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.327955 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.330923 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:56.330925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:56.330929 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:56.330937 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:56.332589 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:56.333037 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:56.333038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:56.333039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:56.333104 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:56.333399 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:56.333400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:56.333400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:56.333530 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:56.333604 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:56.333967 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 05:27:56.334071 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 05:27:56.334409 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.323247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:34357)
0812 05:27:56.323446 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:56.323502 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:56.324287 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:56.324593 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:56.324597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:56.325182 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:56.325486 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.325492 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.326494 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.326495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.327418 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.327420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.327979 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.327980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.328179 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.328953 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:56.328965 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:56.330786 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:56.332688 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:56.332697 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:56.332701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:56.332951 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:56.333177 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:56.333178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:56.333178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:56.333295 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:56.333606 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:56.333614 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:56.333765 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 05:27:56.333944 (+ 179us) client_negotiation.cc:240] Negotiation successful
0812 05:27:56.334119 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260812 05:27:56.334925 7895 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35429
I20260812 05:27:56.335047 7895 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:56.335232 7895 heartbeater.cc:511] Master 127.24.66.126:35429 requested a full tablet report, sending...
I20260812 05:27:56.335700 7556 ts_manager.cc:194] Registered new tserver with Master: ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507)
I20260812 05:27:56.336082 7556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:34357
W20260812 05:27:56.415726 7898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:56.415967 7898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:56.415993 7898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:56.417297 7898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:56.417356 7898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:56.417387 7898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:27:56.419001 7898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:43703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=44635
--tserver_master_addrs=127.24.66.126:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:56.419281 7898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:56.419539 7898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:56.425642 7906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:56.425758 7903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:56.425660 7904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:56.425848 7898 server_base.cc:1034] running on GCE node
I20260812 05:27:56.426077 7898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:56.426427 7898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:56.427601 7898 hybrid_clock.cc:648] HybridClock initialized: now 1786512476427575 us; error 61 us; skew 500 ppm
Aug 12 05:27:56 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512476, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:56.430768 7898 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:27:56.431280 7898 webserver.cc:460] Webserver started at http://127.24.66.67:44635/ using document root <none> and password file <none>
I20260812 05:27:56.431517 7898 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:56.431579 7898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:56.433185 7898 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 05:27:56.433877 7914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:56.434048 7898 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 05:27:56.434144 7898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "9c96585a2d0343c1a89e939f003aae2a"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "e6d8e43ce12c61349d3278ed48ad8a9c"
server_key_iv: "627eea687e77352c293a3972502d356c"
server_key_version: "encryptionkey@0"
I20260812 05:27:56.434566 7898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:56.445353 7898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:56.447849 7898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:56.447971 7898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:56.448177 7898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:56.448513 7898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:56.448549 7898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:56.448572 7898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:56.448617 7898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:56.454826 7898 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:43703
I20260812 05:27:56.454921 8038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:43703 every 8 connection(s)
I20260812 05:27:56.455267 7898 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
Aug 12 05:27:56 dist-test-slave-zpfg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512476, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:27:56.463387 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 7898
I20260812 05:27:56.465448 7757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.456080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:49035 (local address 127.24.66.126:35429)
0812 05:27:56.456167 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:56.456169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:56.456400 (+ 231us) server_negotiation.cc:407] Connection header received
0812 05:27:56.456961 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:56.456963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:56.456976 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:56.457031 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:56.458028 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.458589 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.459614 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.459815 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.459901 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.462110 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:56.462112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:56.462112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:56.462120 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:56.463981 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:56.464412 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:56.464414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:56.464414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:56.464478 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:56.464743 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:56.464744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:56.464745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:56.464880 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:56.464963 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:56.465252 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 05:27:56.465353 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 05:27:56.465700 8041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.456062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:49035)
0812 05:27:56.456273 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:56.456312 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:56.456849 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:56.457101 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:56.457105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:56.457722 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:56.457922 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.457926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.458687 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.458688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.459519 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.459521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.459915 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.459915 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.460051 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.460575 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:56.460583 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:56.462004 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:56.464067 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:56.464077 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:56.464079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:56.464324 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:56.464551 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:56.464552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:56.464553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:56.464671 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:56.464954 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:56.464961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:56.465110 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 05:27:56.465220 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 05:27:56.465397 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260812 05:27:56.466223 8039 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35429
I20260812 05:27:56.466398 8039 heartbeater.cc:464] Registering TS with master...
I20260812 05:27:56.466624 8039 heartbeater.cc:511] Master 127.24.66.126:35429 requested a full tablet report, sending...
I20260812 05:27:56.467139 7556 ts_manager.cc:194] Registered new tserver with Master: 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703)
I20260812 05:27:56.467635 7556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:49035
I20260812 05:27:56.470819 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.463879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42826 (local address 127.24.66.126:35429)
0812 05:27:56.464052 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:56.464054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:56.464064 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:27:56.464139 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:56.464140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:56.464151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:56.464219 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:56.464810 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.465621 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.466686 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.467006 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.467078 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.468481 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:56.468483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:56.468483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:56.468489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:56.469735 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:56.470090 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:56.470091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:56.470091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:56.470146 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:56.470320 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:56.470320 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:56.470320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:56.470488 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:56.470623 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:56.470678 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 05:27:56.470749 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":121,"threads_started":1}
I20260812 05:27:56.471648 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:27:56.474723 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.472027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42840 (local address 127.24.66.126:35429)
0812 05:27:56.472084 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:56.472086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:56.472253 (+ 167us) server_negotiation.cc:407] Connection header received
0812 05:27:56.472335 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:56.472336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:56.472384 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:56.472452 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:27:56.472566 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.473051 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.473839 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.474057 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.474136 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.474331 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:27:56.474487 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:27:56.474590 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:56.474611 (+ 21us) server_negotiation.cc:299] Negotiation successful
0812 05:27:56.474637 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 05:27:56.475051 7556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:27:56.476653 7556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:27:56.479557 7684 tablet_service.cc:1467] Processing CreateTablet for tablet 898e6adbd394444785de0405d006cdc4 (DEFAULT_TABLE table=test-table [id=b38d57f89bd248cc814b281f856dcf40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:27:56.479799 7684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898e6adbd394444785de0405d006cdc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:56.481516 7642 tablet_bootstrap.cc:492] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: Bootstrap starting.
I20260812 05:27:56.481901 7642 tablet_bootstrap.cc:654] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:56.482605 7642 tablet_bootstrap.cc:492] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: No bootstrap required, opened a new log
I20260812 05:27:56.482657 7642 ts_tablet_manager.cc:1397] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:56.483013 7642 raft_consensus.cc:348] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:56.483100 7760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.478996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:43703 (local address 127.0.0.1:40996)
0812 05:27:56.479051 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:56.479062 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:56.479105 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:56.479638 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:56.479639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:56.479641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:56.479692 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.479694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.480846 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.480848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.481724 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.481728 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.482813 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.482814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.482886 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.482911 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 05:27:56.482972 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 05:27:56.483027 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 05:27:56.483110 8046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.479254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.24.66.67:43703)
0812 05:27:56.479465 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:56.479472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:56.479483 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:27:56.479528 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:56.479530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:56.479547 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:56.479634 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:56.479750 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.480738 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.481920 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.482715 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.482815 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.482888 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:56.482988 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 05:27:56.483027 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":124,"threads_started":1}
I20260812 05:27:56.483096 7642 raft_consensus.cc:374] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:56.483155 7642 raft_consensus.cc:729] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd65a69aa01043c3a74e4b17f6deee4c, State: Initialized, Role: FOLLOWER
I20260812 05:27:56.483214 7642 consensus_queue.cc:260] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:56.483315 7642 ts_tablet_manager.cc:1428] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:56.484331 7968 tablet_service.cc:1467] Processing CreateTablet for tablet 898e6adbd394444785de0405d006cdc4 (DEFAULT_TABLE table=test-table [id=b38d57f89bd248cc814b281f856dcf40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:27:56.484663 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.479398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33942 (local address 127.24.66.66:36507)
0812 05:27:56.479615 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:56.479620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:56.479631 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:27:56.480109 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:56.480122 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:56.480175 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:56.480262 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:56.480374 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.481418 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.483161 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.483973 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.484043 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.484096 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:56.484538 (+ 442us) server_negotiation.cc:299] Negotiation successful
0812 05:27:56.484588 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":106,"threads_started":1}
I20260812 05:27:56.484794 7968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898e6adbd394444785de0405d006cdc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:56.484854 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.479054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:36507 (local address 127.0.0.1:33942)
0812 05:27:56.479235 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:56.479246 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:56.479288 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:56.480265 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:56.480266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:56.480269 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:56.480310 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.480312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.481598 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.481663 (+ 65us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.483049 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.483053 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.484265 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.484266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.484342 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.484347 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:27:56.484403 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:27:56.484447 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":123,"threads_started":1}
I20260812 05:27:56.487515 7827 tablet_service.cc:1467] Processing CreateTablet for tablet 898e6adbd394444785de0405d006cdc4 (DEFAULT_TABLE table=test-table [id=b38d57f89bd248cc814b281f856dcf40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:27:56.487946 7827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898e6adbd394444785de0405d006cdc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:27:56.488634 8049 tablet_bootstrap.cc:492] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: Bootstrap starting.
I20260812 05:27:56.489295 8049 tablet_bootstrap.cc:654] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:56.489854 8049 log.cc:824] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:56.490780 8049 tablet_bootstrap.cc:492] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: No bootstrap required, opened a new log
I20260812 05:27:56.490863 8049 ts_tablet_manager.cc:1397] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:27:56.491144 8051 tablet_bootstrap.cc:492] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: Bootstrap starting.
I20260812 05:27:56.491732 8051 tablet_bootstrap.cc:654] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: Neither blocks nor log segments found. Creating new log.
I20260812 05:27:56.492280 8051 log.cc:824] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:56.492938 8049 raft_consensus.cc:348] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:56.493137 8049 raft_consensus.cc:374] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:56.493151 8051 tablet_bootstrap.cc:492] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: No bootstrap required, opened a new log
I20260812 05:27:56.493189 8049 raft_consensus.cc:729] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c96585a2d0343c1a89e939f003aae2a, State: Initialized, Role: FOLLOWER
I20260812 05:27:56.493230 8051 ts_tablet_manager.cc:1397] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:27:56.493335 8049 consensus_queue.cc:260] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:56.493702 8049 ts_tablet_manager.cc:1428] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:27:56.493727 8039 heartbeater.cc:503] Master 127.24.66.126:35429 was elected leader, sending a full tablet report...
I20260812 05:27:56.495244 8051 raft_consensus.cc:348] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:56.495404 8051 raft_consensus.cc:374] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:27:56.495460 8051 raft_consensus.cc:729] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddbb3333d8b74a9e89463bf4663ffeaa, State: Initialized, Role: FOLLOWER
I20260812 05:27:56.495577 8051 consensus_queue.cc:260] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:56.495862 8051 ts_tablet_manager.cc:1428] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:27:56.495944 7895 heartbeater.cc:503] Master 127.24.66.126:35429 was elected leader, sending a full tablet report...
W20260812 05:27:56.573118 7896 tablet.cc:2369] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:27:56.677162 7754 tablet.cc:2369] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:27:56.705964 8040 tablet.cc:2369] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:27:56.732077 8053 raft_consensus.cc:482] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:27:56.732260 8053 raft_consensus.cc:504] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:56.732694 8053 leader_election.cc:288] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507), dd65a69aa01043c3a74e4b17f6deee4c (127.24.66.65:37887)
I20260812 05:27:56.737258 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.733186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:36865 (local address 127.24.66.65:37887)
0812 05:27:56.733352 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:56.733354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:56.733364 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:27:56.733420 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:56.733421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:56.733451 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:56.733515 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:56.733698 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.734471 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.736105 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.736839 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.736912 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.737008 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:56.737121 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 05:27:56.737175 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260812 05:27:56.737334 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.733036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:52419 (local address 127.24.66.66:36507)
0812 05:27:56.733100 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:56.733103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:56.733158 (+ 55us) server_negotiation.cc:407] Connection header received
0812 05:27:56.733192 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:56.733193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:56.733207 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:56.733347 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:27:56.733581 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.734381 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.736105 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.736807 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.736911 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.736921 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:56.737125 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 05:27:56.737264 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 05:27:56.737334 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.733106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:37887 (local address 127.24.66.67:36865)
0812 05:27:56.733292 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:56.733303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:56.733356 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:56.733541 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:56.733543 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:56.733544 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:56.733615 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.733617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.734544 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.734545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.735938 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.735941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.736929 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.736930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.737028 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.737033 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:27:56.737102 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:27:56.737172 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":30,"thread_start_us":132,"threads_started":1}
I20260812 05:27:56.737387 8041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.732957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:36507 (local address 127.24.66.67:52419)
0812 05:27:56.733022 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:56.733036 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:56.733090 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:56.733384 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:56.733385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:56.733387 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:27:56.733472 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.733473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.734468 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.734469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.735938 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:56.735941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.736929 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.736930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:56.737040 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.737043 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:27:56.737102 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 05:27:56.737172 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":76}
I20260812 05:27:56.737789 7847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898e6adbd394444785de0405d006cdc4" candidate_uuid: "9c96585a2d0343c1a89e939f003aae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" is_pre_election: true
I20260812 05:27:56.737789 7706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898e6adbd394444785de0405d006cdc4" candidate_uuid: "9c96585a2d0343c1a89e939f003aae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" is_pre_election: true
I20260812 05:27:56.737987 7847 raft_consensus.cc:2454] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c96585a2d0343c1a89e939f003aae2a in term 0.
I20260812 05:27:56.737994 7706 raft_consensus.cc:2454] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c96585a2d0343c1a89e939f003aae2a in term 0.
I20260812 05:27:56.738274 7919 leader_election.cc:302] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c96585a2d0343c1a89e939f003aae2a, ddbb3333d8b74a9e89463bf4663ffeaa; no voters:
I20260812 05:27:56.738487 8053 raft_consensus.cc:2781] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:27:56.738556 8053 raft_consensus.cc:482] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:27:56.738592 8053 raft_consensus.cc:3037] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:56.739455 8053 raft_consensus.cc:504] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:56.739615 8053 leader_election.cc:288] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [CANDIDATE]: Term 1 election: Requested vote from peers ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507), dd65a69aa01043c3a74e4b17f6deee4c (127.24.66.65:37887)
I20260812 05:27:56.739814 7847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898e6adbd394444785de0405d006cdc4" candidate_uuid: "9c96585a2d0343c1a89e939f003aae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa"
I20260812 05:27:56.739804 7706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898e6adbd394444785de0405d006cdc4" candidate_uuid: "9c96585a2d0343c1a89e939f003aae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd65a69aa01043c3a74e4b17f6deee4c"
I20260812 05:27:56.739928 7847 raft_consensus.cc:3037] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:56.739931 7706 raft_consensus.cc:3037] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:27:56.740718 7706 raft_consensus.cc:2454] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c96585a2d0343c1a89e939f003aae2a in term 1.
I20260812 05:27:56.740912 7847 raft_consensus.cc:2454] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c96585a2d0343c1a89e939f003aae2a in term 1.
I20260812 05:27:56.740949 7919 leader_election.cc:302] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c96585a2d0343c1a89e939f003aae2a, dd65a69aa01043c3a74e4b17f6deee4c; no voters:
I20260812 05:27:56.741147 8053 raft_consensus.cc:2781] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:27:56.741302 8053 raft_consensus.cc:686] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 1 LEADER]: Becoming Leader. State: Replica: 9c96585a2d0343c1a89e939f003aae2a, State: Running, Role: LEADER
I20260812 05:27:56.741420 8053 consensus_queue.cc:237] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:56.742420 7556 catalog_manager.cc:5495] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67). New cstate: current_term: 1 leader_uuid: "9c96585a2d0343c1a89e939f003aae2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } health_report { overall_health: UNKNOWN } } }
I20260812 05:27:56.767589 8046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:56.764736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40998 (local address 127.24.66.67:43703)
0812 05:27:56.764814 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:56.764817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:56.764827 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:27:56.764952 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:56.764954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:56.764977 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:56.765080 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:27:56.765227 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.765761 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.766644 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:56.766889 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:56.766976 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:56.767156 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:27:56.767278 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:27:56.767399 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:56.767429 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 05:27:56.767453 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 05:27:56.769707 7847 raft_consensus.cc:1261] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa [term 1 FOLLOWER]: Refusing update from remote peer 9c96585a2d0343c1a89e939f003aae2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:27:56.769714 7706 raft_consensus.cc:1261] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Refusing update from remote peer 9c96585a2d0343c1a89e939f003aae2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:27:56.770130 8053 consensus_queue.cc:1035] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:27:56.770296 8057 consensus_queue.cc:1035] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:27:56.772605 8060 mvcc.cc:205] Tried to move back new op lower bound from 7317555104845520896 to 7317555104733581312. Current Snapshot: MvccSnapshot[applied={T|T < 7317555104845520896}]
I20260812 05:27:56.779479 7556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:27:56.779608 7556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:27:56.780514 7556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Sending DeleteTablet for 3 replicas of tablet 898e6adbd394444785de0405d006cdc4
I20260812 05:27:56.780929 7968 tablet_service.cc:1514] Processing DeleteTablet for tablet 898e6adbd394444785de0405d006cdc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:27:56 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:40996
I20260812 05:27:56.780997 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 7596
I20260812 05:27:56.781180 8072 tablet_replica.cc:330] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: stopping tablet replica
I20260812 05:27:56.781359 8072 raft_consensus.cc:2229] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:27:56.781471 8072 raft_consensus.cc:2258] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:27:56.783093 7827 tablet_service.cc:1514] Processing DeleteTablet for tablet 898e6adbd394444785de0405d006cdc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:27:56 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:33942
I20260812 05:27:56.783535 8073 tablet_replica.cc:330] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: stopping tablet replica
I20260812 05:27:56.783632 8073 raft_consensus.cc:2229] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:27:56.783721 8073 raft_consensus.cc:2258] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:27:56.784209 8073 ts_tablet_manager.cc:1905] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:27:56.785739 8073 ts_tablet_manager.cc:1918] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:27:56.785800 8073 log.cc:1196] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/898e6adbd394444785de0405d006cdc4
I20260812 05:27:56.786036 8073 ts_tablet_manager.cc:1939] T 898e6adbd394444785de0405d006cdc4 P ddbb3333d8b74a9e89463bf4663ffeaa: Deleting consensus metadata
I20260812 05:27:56.786490 7542 catalog_manager.cc:4841] TS ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507): tablet 898e6adbd394444785de0405d006cdc4 (table test-table [id=b38d57f89bd248cc814b281f856dcf40]) successfully deleted
I20260812 05:27:56.789657 8072 ts_tablet_manager.cc:1905] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:27:56.791703 8072 ts_tablet_manager.cc:1918] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:27:56.791788 8072 log.cc:1196] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/898e6adbd394444785de0405d006cdc4
I20260812 05:27:56.792100 8072 ts_tablet_manager.cc:1939] T 898e6adbd394444785de0405d006cdc4 P 9c96585a2d0343c1a89e939f003aae2a: Deleting consensus metadata
I20260812 05:27:56.792675 7541 catalog_manager.cc:4841] TS 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703): tablet 898e6adbd394444785de0405d006cdc4 (table test-table [id=b38d57f89bd248cc814b281f856dcf40]) successfully deleted
W20260812 05:27:56.797847 7542 connection.cc:535] client connection to 127.24.66.65:37887 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 05:27:56.797994 7542 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 05:27:56.798593 8045 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:27:56.798432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:37887 (local address 127.0.0.1:55844)
0812 05:27:56.798505 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:56.798548 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:37887: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 05:27:56.798928 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 7755
W20260812 05:27:56.798729 7542 catalog_manager.cc:4584] TS dd65a69aa01043c3a74e4b17f6deee4c (127.24.66.65:37887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 898e6adbd394444785de0405d006cdc4: Network error: Client connection negotiation failed: client connection to 127.24.66.65:37887: connect: Connection reset by peer (error 104)
I20260812 05:27:56.808526 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 7898
I20260812 05:27:56.816305 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 7521
I20260812 05:27:56.823513 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35429
--webserver_interface=127.24.66.126
--webserver_port=36095
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:35429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:56.911510 8075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:56.911778 8075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:56.911808 8075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:56.913277 8075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:27:56.913344 8075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:56.913362 8075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:56.913378 8075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:27:56.913391 8075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:27:56.915370 8075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:35429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=36095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:56.915719 8075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:56.916042 8075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:56.922185 8080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:56.922310 8075 server_base.cc:1034] running on GCE node
W20260812 05:27:56.922451 8081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:56.922240 8083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:56.922861 8075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:56.923235 8075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:56.924407 8075 hybrid_clock.cc:648] HybridClock initialized: now 1786512476924392 us; error 26 us; skew 500 ppm
Aug 12 05:27:56 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512476, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:56.927793 8075 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:27:56.928349 8075 webserver.cc:460] Webserver started at http://127.24.66.126:36095/ using document root <none> and password file <none>
I20260812 05:27:56.928617 8075 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:56.928705 8075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:56.930408 8075 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:27:56.931231 8090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:56.931429 8075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:56.931550 8075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "10b8f42a14fd42718a018acb22a3bb3a"
format_stamp: "Formatted at 2026-08-12 05:27:54 on dist-test-slave-zpfg"
server_key: "17e4d4b4eac7bba6699fc29810aca005"
server_key_iv: "4ba5a8aaa3e76e3b66e38a04c27a70da"
server_key_version: "encryptionkey@0"
I20260812 05:27:56.931937 8075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:56.944626 8075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:56.948552 8075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:56.948772 8075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:56.953658 8075 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:35429
I20260812 05:27:56.953701 8143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:35429 every 8 connection(s)
I20260812 05:27:56.954226 8075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:27:56.956014 8144 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:27:56.956852 8144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Bootstrap starting.
I20260812 05:27:56.960117 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 8075
I20260812 05:27:56.960553 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:37887
--local_ip_for_outbound_sockets=127.24.66.65
--tserver_master_addrs=127.24.66.126:35429
--webserver_port=42229
--webserver_interface=127.24.66.65
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:27:56.967031 8144 log.cc:824] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:56.969439 8144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:27:56.969820 8144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a: Bootstrap complete.
I20260812 05:27:56.972582 8144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:56.972822 8144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10b8f42a14fd42718a018acb22a3bb3a, State: Initialized, Role: FOLLOWER
I20260812 05:27:56.972996 8144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:56.973076 8144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:27:56.973150 8144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:27:56.973199 8144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 2 FOLLOWER]: Advancing to term 3
I20260812 05:27:56.974273 8144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:56.974431 8144 leader_election.cc:302] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10b8f42a14fd42718a018acb22a3bb3a; no voters:
I20260812 05:27:56.974663 8144 leader_election.cc:288] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 05:27:56.974704 8148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 3 FOLLOWER]: Leader election won for term 3
I20260812 05:27:56.974825 8148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [term 3 LEADER]: Becoming Leader. State: Replica: 10b8f42a14fd42718a018acb22a3bb3a, State: Running, Role: LEADER
I20260812 05:27:56.974927 8148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } }
I20260812 05:27:56.974963 8144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:27:56.975278 8149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "10b8f42a14fd42718a018acb22a3bb3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } } }
I20260812 05:27:56.975287 8148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10b8f42a14fd42718a018acb22a3bb3a. Latest consensus state: current_term: 3 leader_uuid: "10b8f42a14fd42718a018acb22a3bb3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8f42a14fd42718a018acb22a3bb3a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35429 } } }
I20260812 05:27:56.975389 8149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:56.975400 8148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b8f42a14fd42718a018acb22a3bb3a [sys.catalog]: This master's current role is: LEADER
I20260812 05:27:56.976485 8154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:27:56.977257 8154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:27:56.977394 8154 catalog_manager.cc:1259] Loaded cluster ID: 8f71ab0720564675baac048dc0937cf7
I20260812 05:27:56.977455 8154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 05:27:56.977686 8154 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3b5dfcf9ad google::LogMessage::Fail() at ??:0
@ 0x7f3b5dfd288a google::LogMessage::SendToLog() at ??:0
@ 0x7f3b5dfcf4de google::LogMessage::Flush() at ??:0
@ 0x7f3b5dfd0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3b65ef00f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f3b65ef98b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3b5fe9d0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3b5fe92954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3b60cf66db start_thread at ??:0
@ 0x7f3b5cacd61f clone at ??:0
W20260812 05:27:57.075184 8146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:57.075400 8146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:57.075431 8146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:57.076697 8146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:57.076746 8146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:57.076774 8146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:27:57.078560 8146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:37887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=42229
--tserver_master_addrs=127.24.66.126:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:57.078800 8146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:57.079053 8146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:57.085065 8180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:57.085177 8183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:57.085058 8181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:57.085475 8146 server_base.cc:1034] running on GCE node
I20260812 05:27:57.085747 8146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:57.086023 8146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:57.087205 8146 hybrid_clock.cc:648] HybridClock initialized: now 1786512477087196 us; error 41 us; skew 500 ppm
Aug 12 05:27:57 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512477, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:57.090318 8146 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:27:57.090993 8146 webserver.cc:460] Webserver started at http://127.24.66.65:42229/ using document root <none> and password file <none>
I20260812 05:27:57.091253 8146 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:57.091331 8146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:57.092952 8146 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:27:57.093787 8191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:57.094003 8146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:27:57.094122 8146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "dd65a69aa01043c3a74e4b17f6deee4c"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "b9d3af8120ade76fdf584d1e510cf57d"
server_key_iv: "4203a831127542bbf83b11c1b7a7e809"
server_key_version: "encryptionkey@0"
I20260812 05:27:57.094578 8146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:57.115868 8146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:57.118635 8146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:57.118854 8146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:57.119143 8146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:57.119642 8199 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 05:27:57.121964 8146 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 05:27:57.122044 8146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260812 05:27:57.122099 8146 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 05:27:57.122953 8146 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 05:27:57.123021 8146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 05:27:57.123106 8199 tablet_bootstrap.cc:492] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: Bootstrap starting.
I20260812 05:27:57.131349 8146 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:37887
I20260812 05:27:57.131798 8146 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:27:57.132251 8311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:37887 every 8 connection(s)
I20260812 05:27:57.135768 8199 log.cc:824] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:27:57.137708 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 8146
I20260812 05:27:57.138043 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:36507
--local_ip_for_outbound_sockets=127.24.66.66
--tserver_master_addrs=127.24.66.126:35429
--webserver_port=40027
--webserver_interface=127.24.66.66
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:27:57.139300 8199 tablet_bootstrap.cc:492] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:27:57.139684 8199 tablet_bootstrap.cc:492] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: Bootstrap complete.
I20260812 05:27:57.141191 8199 ts_tablet_manager.cc:1397] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.010s
I20260812 05:27:57.143296 8199 raft_consensus.cc:348] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:57.143754 8199 raft_consensus.cc:729] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd65a69aa01043c3a74e4b17f6deee4c, State: Initialized, Role: FOLLOWER
I20260812 05:27:57.143939 8199 consensus_queue.cc:260] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:57.144299 8199 ts_tablet_manager.cc:1428] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
W20260812 05:27:57.204797 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:27:57.132649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:59019)
0812 05:27:57.132931 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:57.132980 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:57.133506 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:57.204505 (+ 70999us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":169,"threads_started":1}
W20260812 05:27:57.205372 8312 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:35429 (0 consecutive failures): Network error: Failed to ping master at 127.24.66.126:35429: Client connection negotiation failed: client connection to 127.24.66.126:35429: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 05:27:57.205941 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:57.205778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:39725)
0812 05:27:57.205847 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:57.205907 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 05:27:57.206463 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:57.206314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:49567)
0812 05:27:57.206406 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:57.206439 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260812 05:27:57.206631 8312 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:27:57.232604 8317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:57.232851 8317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:57.232878 8317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:57.234198 8317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:57.234256 8317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:57.234287 8317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:27:57.236111 8317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:36507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=40027
--tserver_master_addrs=127.24.66.126:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:57.236402 8317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:57.236672 8317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:57.242765 8327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:57.242803 8323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:57.242751 8325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:57.242835 8317 server_base.cc:1034] running on GCE node
I20260812 05:27:57.243228 8317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:57.243541 8317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:57.244722 8317 hybrid_clock.cc:648] HybridClock initialized: now 1786512477244702 us; error 43 us; skew 500 ppm
Aug 12 05:27:57 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512477, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:57.247643 8317 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:27:57.248140 8317 webserver.cc:460] Webserver started at http://127.24.66.66:40027/ using document root <none> and password file <none>
I20260812 05:27:57.248420 8317 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:57.248472 8317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:57.250319 8317 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:27:57.251060 8334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:57.251271 8317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:57.251371 8317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "ddbb3333d8b74a9e89463bf4663ffeaa"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "9ab6d0fcf8a5422e7970944193b0ac95"
server_key_iv: "f12717edcfff4c67b695dab49b65cc16"
server_key_version: "encryptionkey@0"
I20260812 05:27:57.251746 8317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:57.260290 8317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:57.262619 8317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:57.262801 8317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:57.263111 8317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:57.263463 8317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:57.263525 8317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:57.263566 8317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:57.263595 8317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:57.270848 8317 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:36507
I20260812 05:27:57.270915 8451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:36507 every 8 connection(s)
I20260812 05:27:57.271335 8317 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:27:57.272584 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:57.272157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:46637)
0812 05:27:57.272336 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:57.272426 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
W20260812 05:27:57.273100 8452 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:35429 (0 consecutive failures): Network error: Failed to ping master at 127.24.66.126:35429: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
I20260812 05:27:57.273424 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:57.273325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:56795)
0812 05:27:57.273376 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:57.273405 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 05:27:57.273854 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:57.273768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:52111)
0812 05:27:57.273813 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:57.273837 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 05:27:57.273991 8452 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 05:27:57.276569 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 8317
I20260812 05:27:57.276861 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:43703
--local_ip_for_outbound_sockets=127.24.66.67
--tserver_master_addrs=127.24.66.126:35429
--webserver_port=44635
--webserver_interface=127.24.66.67
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:27:57.365175 8456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:27:57.365437 8456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:27:57.365466 8456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:27:57.366631 8456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:27:57.366669 8456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:27:57.366696 8456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:27:57.368108 8456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:43703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=44635
--tserver_master_addrs=127.24.66.126:35429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:27:57.368301 8456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:27:57.368491 8456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:27:57.373782 8461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:57.373826 8465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:27:57.373798 8462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:27:57.374191 8456 server_base.cc:1034] running on GCE node
I20260812 05:27:57.374454 8456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:27:57.374681 8456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:27:57.375859 8456 hybrid_clock.cc:648] HybridClock initialized: now 1786512477375849 us; error 41 us; skew 500 ppm
Aug 12 05:27:57 dist-test-slave-zpfg krb5kdc[6937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512477, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:27:57.378546 8456 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:27:57.379065 8456 webserver.cc:460] Webserver started at http://127.24.66.67:44635/ using document root <none> and password file <none>
I20260812 05:27:57.379307 8456 fs_manager.cc:359] Metadata directory not provided
I20260812 05:27:57.379352 8456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:27:57.380833 8456 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:57.381511 8472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:57.381745 8456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:27:57.381825 8456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "9c96585a2d0343c1a89e939f003aae2a"
format_stamp: "Formatted at 2026-08-12 05:27:55 on dist-test-slave-zpfg"
server_key: "e6d8e43ce12c61349d3278ed48ad8a9c"
server_key_iv: "627eea687e77352c293a3972502d356c"
server_key_version: "encryptionkey@0"
I20260812 05:27:57.382179 8456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:27:57.395390 8456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:27:57.397634 8456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:27:57.397755 8456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:27:57.397979 8456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:27:57.398327 8456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:27:57.398422 8456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:57.398470 8456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:27:57.398499 8456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:27:57.404816 8456 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:43703
I20260812 05:27:57.404902 8588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:43703 every 8 connection(s)
I20260812 05:27:57.405277 8456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:27:57.406623 8592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:57.406098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:53053)
0812 05:27:57.406272 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:57.406473 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
W20260812 05:27:57.407021 8589 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:35429 (0 consecutive failures): Network error: Failed to ping master at 127.24.66.126:35429: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
I20260812 05:27:57.407330 8592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:57.407221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:42875)
0812 05:27:57.407274 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:57.407305 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 05:27:57.407626 8592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:57.407536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:53937)
0812 05:27:57.407580 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:57.407603 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 05:27:57.407727 8589 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 05:27:57.413851 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 8456
I20260812 05:27:58.207356 8594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:58.207060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:59793)
0812 05:27:58.207228 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:58.207283 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":72,"threads_started":1}
I20260812 05:27:58.274742 8595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:58.274494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:38499)
0812 05:27:58.274634 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:58.274680 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260812 05:27:58.408310 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:58.408057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:38971)
0812 05:27:58.408207 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:58.408249 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260812 05:27:58.953403 8597 raft_consensus.cc:482] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:27:58.953576 8597 raft_consensus.cc:504] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:27:58.954028 8597 leader_election.cc:288] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507), 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703)
Aug 12 05:27:58 dist-test-slave-zpfg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512477, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.66@KRBTEST.COM
Aug 12 05:27:58 dist-test-slave-zpfg krb5kdc[6937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512477, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.67@KRBTEST.COM
I20260812 05:27:58.967204 8601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:58.954464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:52537 (local address 127.24.66.67:43703)
0812 05:27:58.954616 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:58.954630 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:58.954691 (+ 61us) server_negotiation.cc:407] Connection header received
0812 05:27:58.955359 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:58.955363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:58.955427 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:58.955588 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:58.956990 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:58.958242 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:58.959536 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:58.960087 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:58.960369 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:58.963281 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:58.963286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:58.963286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:58.963304 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:58.965295 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:58.965770 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:58.965773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:58.965773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:58.965865 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:58.966199 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:58.966200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:58.966200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:58.966512 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:58.966605 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:58.966925 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 05:27:58.967053 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
I20260812 05:27:58.967180 8598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:58.954219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:36507 (local address 127.24.66.65:41731)
0812 05:27:58.954366 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:58.954382 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:58.954455 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:58.955795 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:58.955798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:58.956483 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:58.956808 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:58.956817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:58.958313 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:58.958315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:58.959422 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:58.959426 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:58.960251 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:58.960252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:58.960394 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:58.961066 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:58.961075 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:58.962717 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:58.965315 (+ 2598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:58.965325 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:58.965328 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:58.965660 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:58.965984 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:58.965985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:58.965985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:58.966083 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:58.966609 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:58.966610 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:58.966744 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 05:27:58.966853 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 05:27:58.967061 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":34,"thread_start_us":77,"threads_started":1}
I20260812 05:27:58.967240 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:58.954421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:41731 (local address 127.24.66.66:36507)
0812 05:27:58.954673 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 05:27:58.954681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:27:58.954701 (+ 20us) server_negotiation.cc:407] Connection header received
0812 05:27:58.955360 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:27:58.955363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:27:58.955427 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:27:58.955588 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:27:58.956967 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:58.958178 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:58.959522 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:27:58.960146 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:27:58.960276 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:58.962904 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:27:58.962914 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:27:58.962915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:27:58.962948 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:27:58.965130 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:58.965771 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:58.965781 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:58.965782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:58.965878 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:27:58.966167 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:27:58.966169 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:27:58.966169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:27:58.966515 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:27:58.966620 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 05:27:58.966931 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 05:27:58.967105 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":66,"threads_started":1}
I20260812 05:27:58.967322 8599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:58.954280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:43703 (local address 127.24.66.65:52537)
0812 05:27:58.954584 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:58.954596 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:27:58.954641 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:27:58.955795 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:27:58.955798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:27:58.956477 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:27:58.956808 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:58.956817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:58.958392 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:58.958393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:58.959423 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:27:58.959426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:27:58.960454 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:27:58.960455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:27:58.960503 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:27:58.961251 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:27:58.961257 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:27:58.963149 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:27:58.965400 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:58.965401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:58.965402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:58.965662 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:58.965943 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:27:58.965945 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:27:58.965945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:27:58.966088 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:27:58.966598 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:27:58.966606 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:27:58.966745 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 05:27:58.966854 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 05:27:58.967152 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":181,"thread_start_us":114,"threads_started":1}
W20260812 05:27:58.968158 8196 leader_election.cc:341] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507): Not found: Tablet not found: 898e6adbd394444785de0405d006cdc4
W20260812 05:27:58.968705 8195 leader_election.cc:341] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703): Not found: Tablet not found: 898e6adbd394444785de0405d006cdc4
I20260812 05:27:58.968753 8195 leader_election.cc:302] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd65a69aa01043c3a74e4b17f6deee4c; no voters: 9c96585a2d0343c1a89e939f003aae2a, ddbb3333d8b74a9e89463bf4663ffeaa
I20260812 05:27:58.968869 8597 raft_consensus.cc:2726] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:27:59.208072 8598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:59.207889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:46313)
0812 05:27:59.207980 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:59.208028 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 05:27:59.275599 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:59.275305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:60043)
0812 05:27:59.275482 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:59.275540 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260812 05:27:59.409117 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:27:59.408832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:45785)
0812 05:27:59.409003 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 05:27:59.409061 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260812 05:28:00.209111 8605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:00.208809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:44883)
0812 05:28:00.208997 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:00.209054 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":81,"threads_started":1}
I20260812 05:28:00.276531 8606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:00.276229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:34859)
0812 05:28:00.276429 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:00.276479 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":83,"threads_started":1}
I20260812 05:28:00.409885 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:00.409628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:45921)
0812 05:28:00.409771 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:00.409830 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260812 05:28:01.149529 8608 raft_consensus.cc:482] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:28:01.149662 8608 raft_consensus.cc:504] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:28:01.149920 8608 leader_election.cc:288] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507), 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703)
W20260812 05:28:01.150568 8196 leader_election.cc:341] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507): Not found: Tablet not found: 898e6adbd394444785de0405d006cdc4
W20260812 05:28:01.150645 8195 leader_election.cc:341] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703): Not found: Tablet not found: 898e6adbd394444785de0405d006cdc4
I20260812 05:28:01.150681 8196 leader_election.cc:302] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd65a69aa01043c3a74e4b17f6deee4c; no voters: 9c96585a2d0343c1a89e939f003aae2a, ddbb3333d8b74a9e89463bf4663ffeaa
I20260812 05:28:01.150794 8608 raft_consensus.cc:2726] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:28:01.210141 8609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:01.209839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:33871)
0812 05:28:01.210018 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:01.210069 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":77,"threads_started":1}
I20260812 05:28:01.277354 8610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:01.277091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:41415)
0812 05:28:01.277246 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:01.277286 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260812 05:28:01.410729 8611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:01.410484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:40019)
0812 05:28:01.410616 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:01.410662 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260812 05:28:02.211057 8612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:02.210811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:37293)
0812 05:28:02.210956 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:02.211004 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260812 05:28:02.278155 8613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:02.277917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:37177)
0812 05:28:02.278058 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:02.278106 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
I20260812 05:28:02.411631 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:02.411349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:59695)
0812 05:28:02.411531 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:02.411580 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260812 05:28:03.211931 8615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:03.211662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:49621)
0812 05:28:03.211803 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:03.211881 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260812 05:28:03.279066 8616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:03.278781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:55543)
0812 05:28:03.278966 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:03.279014 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":105,"threads_started":1}
I20260812 05:28:03.412468 8617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:03.412202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:36043)
0812 05:28:03.412374 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:03.412420 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":79,"threads_started":1}
I20260812 05:28:04.212667 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:04.212439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:33867)
0812 05:28:04.212580 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:04.212618 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260812 05:28:04.279943 8619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:04.279681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:48915)
0812 05:28:04.279845 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:04.279893 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260812 05:28:04.374569 8620 raft_consensus.cc:482] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:28:04.374696 8620 raft_consensus.cc:504] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:28:04.374902 8620 leader_election.cc:288] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507), 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703)
W20260812 05:28:04.375556 8195 leader_election.cc:341] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703): Not found: Tablet not found: 898e6adbd394444785de0405d006cdc4
W20260812 05:28:04.375645 8196 leader_election.cc:341] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507): Not found: Tablet not found: 898e6adbd394444785de0405d006cdc4
I20260812 05:28:04.375706 8196 leader_election.cc:302] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd65a69aa01043c3a74e4b17f6deee4c; no voters: 9c96585a2d0343c1a89e939f003aae2a, ddbb3333d8b74a9e89463bf4663ffeaa
I20260812 05:28:04.375808 8620 raft_consensus.cc:2726] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:28:04.413331 8621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:04.413065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:39365)
0812 05:28:04.413235 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:04.413275 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":69,"threads_started":1}
I20260812 05:28:05.213610 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:05.213335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:47287)
0812 05:28:05.213507 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:05.213556 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
I20260812 05:28:05.280941 8623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:05.280698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:36429)
0812 05:28:05.280844 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:05.280889 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260812 05:28:05.414324 8624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:05.414056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:55719)
0812 05:28:05.414226 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:05.414273 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260812 05:28:06.214555 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:06.214203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:39717)
0812 05:28:06.214460 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:06.214504 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":73,"threads_started":1}
I20260812 05:28:06.281708 8626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:06.281475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:60089)
0812 05:28:06.281611 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:06.281659 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260812 05:28:06.415270 8627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:06.415012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:51891)
0812 05:28:06.415173 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:06.415219 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260812 05:28:06.558482 8628 raft_consensus.cc:482] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:28:06.558600 8628 raft_consensus.cc:504] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddbb3333d8b74a9e89463bf4663ffeaa" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36507 } } peers { permanent_uuid: "9c96585a2d0343c1a89e939f003aae2a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 43703 } } peers { permanent_uuid: "dd65a69aa01043c3a74e4b17f6deee4c" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 37887 } }
I20260812 05:28:06.558842 8628 leader_election.cc:288] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507), 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703)
W20260812 05:28:06.559260 8196 leader_election.cc:341] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ddbb3333d8b74a9e89463bf4663ffeaa (127.24.66.66:36507): Not found: Tablet not found: 898e6adbd394444785de0405d006cdc4
W20260812 05:28:06.559342 8195 leader_election.cc:341] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9c96585a2d0343c1a89e939f003aae2a (127.24.66.67:43703): Not found: Tablet not found: 898e6adbd394444785de0405d006cdc4
I20260812 05:28:06.559392 8195 leader_election.cc:302] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd65a69aa01043c3a74e4b17f6deee4c; no voters: 9c96585a2d0343c1a89e939f003aae2a, ddbb3333d8b74a9e89463bf4663ffeaa
I20260812 05:28:06.559476 8628 raft_consensus.cc:2726] T 898e6adbd394444785de0405d006cdc4 P dd65a69aa01043c3a74e4b17f6deee4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:28:07.215384 8629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:07.215114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:56625)
0812 05:28:07.215286 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:07.215335 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":67,"threads_started":1}
I20260812 05:28:07.282665 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:07.282303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:47613)
0812 05:28:07.282529 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:07.282578 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":121,"threads_started":1}
I20260812 05:28:07.416194 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:07.415793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:43405)
0812 05:28:07.416010 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:07.416069 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":130,"threads_started":1}
I20260812 05:28:08.216269 8632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:08.216006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:50005)
0812 05:28:08.216160 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:08.216210 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260812 05:28:08.283560 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:08.283249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:45665)
0812 05:28:08.283404 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:08.283483 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":124,"threads_started":1}
I20260812 05:28:08.417114 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:08.416860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:55015)
0812 05:28:08.417017 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:08.417062 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
I20260812 05:28:09.217317 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:09.217011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:58479)
0812 05:28:09.217207 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:09.217261 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":96,"threads_started":1}
I20260812 05:28:09.284422 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:09.284056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:58649)
0812 05:28:09.284283 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:09.284335 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":130,"threads_started":1}
I20260812 05:28:09.418133 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:09.417870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:36993)
0812 05:28:09.418036 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:09.418082 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":84,"threads_started":1}
I20260812 05:28:10.218218 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:10.217971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:56217)
0812 05:28:10.218124 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:10.218170 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
I20260812 05:28:10.285339 8639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:10.285065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:48197)
0812 05:28:10.285218 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:10.285274 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260812 05:28:10.419117 8640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:10.418831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:38723)
0812 05:28:10.419019 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:10.419064 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":90,"threads_started":1}
I20260812 05:28:11.219306 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:11.218909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:50807)
0812 05:28:11.219108 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:11.219255 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":97,"threads_started":1}
I20260812 05:28:11.286301 8642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:11.286027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:49133)
0812 05:28:11.286196 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:11.286247 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
I20260812 05:28:11.420004 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:11.419753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:53463)
0812 05:28:11.419907 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:11.419952 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260812 05:28:12.220173 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:12.219931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.65:57513)
0812 05:28:12.220084 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:12.220124 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260812 05:28:12.287264 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:12.287005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.66:49681)
0812 05:28:12.287170 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:12.287215 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260812 05:28:12.421180 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:12.420885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35429 (local address 127.24.66.67:53667)
0812 05:28:12.421078 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:12.421125 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":85,"threads_started":1}
I20260812 05:28:12.422802 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 8146
I20260812 05:28:12.430811 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 8317
I20260812 05:28:12.437216 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 8456
2026-08-12T05:28:12Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18917 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:28:12 dist-test-slave-zpfg krb5kdc[8650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:28:12 dist-test-slave-zpfg krb5kdc[8650](info): set up 2 sockets
Aug 12 05:28:12 dist-test-slave-zpfg krb5kdc[8650](info): commencing operation
krb5kdc: starting...
W20260812 05:28:13.639837 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:28:13 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:28:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:28:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:28:13.779587 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35627
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:35627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:28:13.872462 8664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:13.872740 8664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:13.872785 8664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:13.874012 8664 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 05:28:13.874097 8664 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 05:28:13.874128 8664 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 05:28:13.874148 8664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:28:13.874181 8664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:13.874208 8664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:13.874236 8664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:28:13.874265 8664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:28:13.875988 8664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:35627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:13.876339 8664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:13.876602 8664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:28:13.882809 8664 server_base.cc:1034] running on GCE node
W20260812 05:28:13.882784 8670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:13.882843 8669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:13.882769 8673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:13.883270 8664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:13.883641 8664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:13.884804 8664 hybrid_clock.cc:648] HybridClock initialized: now 1786512493884779 us; error 40 us; skew 500 ppm
Aug 12 05:28:13 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512493, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:13.887827 8664 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:28:13.888314 8664 webserver.cc:460] Webserver started at http://127.24.66.126:37955/ using document root <none> and password file <none>
I20260812 05:28:13.888523 8664 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:13.888571 8664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:13.888688 8664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:28:13.889896 8664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c"
format_stamp: "Formatted at 2026-08-12 05:28:13 on dist-test-slave-zpfg"
server_key: "53879f3fd81700ddc2456776b4b98b8f"
server_key_iv: "4b2146bae20660cdfb5ddcc5f108629a"
server_key_version: "encryptionkey@0"
I20260812 05:28:13.890249 8664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c"
format_stamp: "Formatted at 2026-08-12 05:28:13 on dist-test-slave-zpfg"
server_key: "53879f3fd81700ddc2456776b4b98b8f"
server_key_iv: "4b2146bae20660cdfb5ddcc5f108629a"
server_key_version: "encryptionkey@0"
I20260812 05:28:13.891892 8664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:28:13.892773 8680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:13.893029 8664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:28:13.893132 8664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c"
format_stamp: "Formatted at 2026-08-12 05:28:13 on dist-test-slave-zpfg"
server_key: "53879f3fd81700ddc2456776b4b98b8f"
server_key_iv: "4b2146bae20660cdfb5ddcc5f108629a"
server_key_version: "encryptionkey@0"
I20260812 05:28:13.893208 8664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:13.908753 8664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:13.911908 8664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:13.912056 8664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:13.917124 8664 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:35627
I20260812 05:28:13.917160 8734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:35627 every 8 connection(s)
I20260812 05:28:13.917636 8664 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:28:13.918826 8735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:28:13.921160 8735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Bootstrap starting.
I20260812 05:28:13.921736 8735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Neither blocks nor log segments found. Creating new log.
I20260812 05:28:13.921999 8735 log.cc:824] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:28:13.922775 8735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: No bootstrap required, opened a new log
I20260812 05:28:13.924383 8735 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:13.924535 8735 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:28:13.924558 8735 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8af8e0f532b4ed7bf2f9888c4b9547c, State: Initialized, Role: FOLLOWER
I20260812 05:28:13.924712 8735 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:13.924800 8735 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:28:13.924823 8735 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:28:13.924854 8735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:28:13.925401 8735 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:13.925491 8735 leader_election.cc:302] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8af8e0f532b4ed7bf2f9888c4b9547c; no voters:
I20260812 05:28:13.925639 8735 leader_election.cc:288] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:28:13.925711 8738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:28:13.925971 8738 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 1 LEADER]: Becoming Leader. State: Replica: b8af8e0f532b4ed7bf2f9888c4b9547c, State: Running, Role: LEADER
I20260812 05:28:13.925974 8735 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:28:13.926085 8738 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:13.926532 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 8664
I20260812 05:28:13.926663 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:28:13.926918 24841 external_mini_cluster.cc:1428] Setting key 79adb515f23d2af7e86f4d5c9e93a1a5
I20260812 05:28:13.927150 8740 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8af8e0f532b4ed7bf2f9888c4b9547c. Latest consensus state: current_term: 1 leader_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } } }
I20260812 05:28:13.927254 8740 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: This master's current role is: LEADER
I20260812 05:28:13.927667 8739 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } } }
I20260812 05:28:13.928035 8739 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: This master's current role is: LEADER
W20260812 05:28:13.928210 8755 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:28:13.928277 8755 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:28:13.928323 8752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:28:13.928484 8752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:28:13.930208 8752 catalog_manager.cc:1347] Generated new cluster ID: 33d9025370634444ba0654666e7f3e87
I20260812 05:28:13.930265 8752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:28:13 dist-test-slave-zpfg krb5kdc[8650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:28:13.936996 8752 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:28:13.937556 8752 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:28:13.940325 8752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Generated new TSK 0
I20260812 05:28:13.940515 8752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:28:13.941262 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:13.927962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48652 (local address 127.24.66.126:35627)
0812 05:28:13.928254 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:13.928262 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:13.928325 (+ 63us) server_negotiation.cc:407] Connection header received
0812 05:28:13.928454 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:13.928468 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:13.928533 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:13.928718 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:13.929494 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:13.930941 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:13.932049 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:13.932739 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:13.932915 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:13.935991 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:13.935996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:13.935996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:13.936010 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:13.939368 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:13.939844 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:13.939848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:13.939848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:13.939970 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:13.940236 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:13.940238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:13.940238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:13.940574 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:13.940740 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:13.940843 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 05:28:13.941017 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:28:13.974287 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35627
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:28:14.094422 8762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:14.094666 8762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:14.094707 8762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:14.095925 8762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:14.095994 8762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:14.096041 8762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:28:14.097762 8762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:14.098043 8762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:14.098312 8762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:28:14.103830 8762 server_base.cc:1034] running on GCE node
W20260812 05:28:14.103797 8767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:14.103914 8768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:14.103780 8770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:14.104249 8762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:14.104524 8762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:14.105731 8762 hybrid_clock.cc:648] HybridClock initialized: now 1786512494105714 us; error 31 us; skew 500 ppm
Aug 12 05:28:14 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512494, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:14.108583 8762 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:28:14.109073 8762 webserver.cc:460] Webserver started at http://127.24.66.65:38659/ using document root <none> and password file <none>
I20260812 05:28:14.109338 8762 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:14.109411 8762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:14.109560 8762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:28:14.110747 8762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "cb207717cd0f40debf4ef346b3425be9"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "bb4bec58c8167604d729b852f34c33a2"
server_key_iv: "e822680154559b72f79a869861c65ba5"
server_key_version: "encryptionkey@0"
I20260812 05:28:14.111119 8762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "cb207717cd0f40debf4ef346b3425be9"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "bb4bec58c8167604d729b852f34c33a2"
server_key_iv: "e822680154559b72f79a869861c65ba5"
server_key_version: "encryptionkey@0"
I20260812 05:28:14.112679 8762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:28:14.113612 8778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:14.113885 8762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:14.114001 8762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "cb207717cd0f40debf4ef346b3425be9"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "bb4bec58c8167604d729b852f34c33a2"
server_key_iv: "e822680154559b72f79a869861c65ba5"
server_key_version: "encryptionkey@0"
I20260812 05:28:14.114080 8762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:14.125953 8762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:14.128418 8762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:14.128592 8762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:14.128866 8762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:28:14.129207 8762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:28:14.129264 8762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:14.129302 8762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:28:14.129354 8762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:14.135720 8762 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:34163
I20260812 05:28:14.135880 8898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:34163 every 8 connection(s)
I20260812 05:28:14.136186 8762 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:28:14.142079 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 8762
I20260812 05:28:14.142378 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:28:14.142627 24841 external_mini_cluster.cc:1428] Setting key 9161c672e23c5c2efd039278d9661988
Aug 12 05:28:14 dist-test-slave-zpfg krb5kdc[8650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512494, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:28:14.148250 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.137130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:33215 (local address 127.24.66.126:35627)
0812 05:28:14.137214 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:14.137216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:14.137505 (+ 289us) server_negotiation.cc:407] Connection header received
0812 05:28:14.138054 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:14.138055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:14.138065 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:14.138118 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:14.139355 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.139953 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.141138 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.141614 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.141686 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.144421 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:14.144422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:14.144422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:14.144433 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:14.146243 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:14.146899 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:14.146901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:14.146901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:14.146972 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:14.147334 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:14.147335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:14.147335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:14.147534 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:14.147593 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:14.148009 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 05:28:14.148115 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:28:14.148451 8901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.137111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:33215)
0812 05:28:14.137356 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:14.137401 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:14.137944 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:14.138216 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:14.138221 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:14.138985 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:28:14.139217 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.139224 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.140089 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.140090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.141025 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.141028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.141718 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.141719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.141845 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.142601 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:28:14.142617 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:28:14.144313 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:28:14.146385 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:14.146397 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:14.146400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:14.146784 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:14.147058 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:14.147059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:14.147059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:14.147230 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:14.147603 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:28:14.147611 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:28:14.147836 (+ 225us) client_negotiation.cc:769] Sending connection context
0812 05:28:14.147953 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 05:28:14.148154 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":115,"threads_started":1}
I20260812 05:28:14.149125 8899 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35627
I20260812 05:28:14.149256 8899 heartbeater.cc:464] Registering TS with master...
I20260812 05:28:14.149857 8899 heartbeater.cc:511] Master 127.24.66.126:35627 requested a full tablet report, sending...
I20260812 05:28:14.150710 8697 ts_manager.cc:194] Registered new tserver with Master: cb207717cd0f40debf4ef346b3425be9 (127.24.66.65:34163)
I20260812 05:28:14.151508 8697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:33215
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:28:14.177132 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35627
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:28:14.264868 8904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:14.265092 8904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:14.265117 8904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:14.266379 8904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:14.266428 8904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:14.266455 8904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:28:14.267988 8904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:14.268267 8904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:14.268515 8904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:28:14.274277 8910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:14.274294 8912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:14.274298 8909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:14.274505 8904 server_base.cc:1034] running on GCE node
I20260812 05:28:14.274789 8904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:14.275076 8904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:14.276350 8904 hybrid_clock.cc:648] HybridClock initialized: now 1786512494276328 us; error 53 us; skew 500 ppm
Aug 12 05:28:14 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512494, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:14.279491 8904 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:28:14.279985 8904 webserver.cc:460] Webserver started at http://127.24.66.66:39079/ using document root <none> and password file <none>
I20260812 05:28:14.280283 8904 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:14.280361 8904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:14.280525 8904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:28:14.281674 8904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "35c78683293248f08e8013fcaac49603"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "e015060a89cc03faae50091afd40a366"
server_key_iv: "cdb927752152df0a887112eb7165eb57"
server_key_version: "encryptionkey@0"
I20260812 05:28:14.282078 8904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "35c78683293248f08e8013fcaac49603"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "e015060a89cc03faae50091afd40a366"
server_key_iv: "cdb927752152df0a887112eb7165eb57"
server_key_version: "encryptionkey@0"
I20260812 05:28:14.283670 8904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:28:14.284615 8919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:14.284925 8904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:14.285079 8904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "35c78683293248f08e8013fcaac49603"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "e015060a89cc03faae50091afd40a366"
server_key_iv: "cdb927752152df0a887112eb7165eb57"
server_key_version: "encryptionkey@0"
I20260812 05:28:14.285224 8904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:14.303007 8904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:14.305341 8904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:14.305527 8904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:14.305799 8904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:28:14.306135 8904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:28:14.306193 8904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:14.306313 8904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:28:14.306372 8904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:14.312737 8904 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:37919
I20260812 05:28:14.312829 9035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:37919 every 8 connection(s)
I20260812 05:28:14.313184 8904 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:28:14.314004 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 8904
I20260812 05:28:14.314141 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:28:14.314404 24841 external_mini_cluster.cc:1428] Setting key ca3f2c20a3e629d0847a2330d76a894c
Aug 12 05:28:14 dist-test-slave-zpfg krb5kdc[8650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512494, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
I20260812 05:28:14.324743 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.314101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:49625 (local address 127.24.66.126:35627)
0812 05:28:14.314159 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:14.314162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:14.314499 (+ 337us) server_negotiation.cc:407] Connection header received
0812 05:28:14.315108 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:14.315110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:14.315122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:14.315169 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:14.316215 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.316829 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.318238 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.318613 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.318689 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.321395 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:14.321396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:14.321397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:14.321405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:14.323203 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:14.323656 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:14.323657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:14.323658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:14.323734 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:14.323986 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:14.323987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:14.323988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:14.324135 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:14.324210 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:14.324523 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 05:28:14.324618 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 05:28:14.324872 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.314050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:49625)
0812 05:28:14.314331 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:14.314413 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:14.315001 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:14.315254 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:14.315258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:14.315874 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:28:14.316077 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.316082 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.316974 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.316975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.318144 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.318147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.318696 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.318697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.318820 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.319501 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:28:14.319515 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:28:14.321267 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:28:14.323314 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:14.323321 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:14.323323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:14.323560 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:14.323809 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:14.323810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:14.323810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:14.323919 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:14.324201 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:28:14.324205 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:28:14.324387 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 05:28:14.324476 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 05:28:14.324610 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":92,"threads_started":1}
I20260812 05:28:14.325424 9036 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35627
I20260812 05:28:14.325532 9036 heartbeater.cc:464] Registering TS with master...
I20260812 05:28:14.325729 9036 heartbeater.cc:511] Master 127.24.66.126:35627 requested a full tablet report, sending...
I20260812 05:28:14.326243 8697 ts_manager.cc:194] Registered new tserver with Master: 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919)
I20260812 05:28:14.326676 8697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:49625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:28:14.343600 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35627
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:28:14.438220 9041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:14.438539 9041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:14.438571 9041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:14.439884 9041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:14.439930 9041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:14.439960 9041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:28:14.441880 9041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:14.442173 9041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:14.442512 9041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:28:14.448386 9046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:14.448386 9049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:14.448603 9041 server_base.cc:1034] running on GCE node
W20260812 05:28:14.448755 9047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:14.449019 9041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:14.449339 9041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:14.450523 9041 hybrid_clock.cc:648] HybridClock initialized: now 1786512494450499 us; error 40 us; skew 500 ppm
Aug 12 05:28:14 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512494, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:14.453936 9041 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:28:14.454526 9041 webserver.cc:460] Webserver started at http://127.24.66.67:37329/ using document root <none> and password file <none>
I20260812 05:28:14.454771 9041 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:14.454856 9041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:14.455001 9041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:28:14.456142 9041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "cce176cdb26c41ce977b9428c09d9335"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "a0444fd5b97526232384eea55fb34d41"
server_key_iv: "fb61ff593058d908757ca97a7b5ef5e8"
server_key_version: "encryptionkey@0"
I20260812 05:28:14.456570 9041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "cce176cdb26c41ce977b9428c09d9335"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "a0444fd5b97526232384eea55fb34d41"
server_key_iv: "fb61ff593058d908757ca97a7b5ef5e8"
server_key_version: "encryptionkey@0"
I20260812 05:28:14.463021 9041 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.000s
I20260812 05:28:14.464134 9056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:14.464326 9041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:14.464409 9041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "cce176cdb26c41ce977b9428c09d9335"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "a0444fd5b97526232384eea55fb34d41"
server_key_iv: "fb61ff593058d908757ca97a7b5ef5e8"
server_key_version: "encryptionkey@0"
I20260812 05:28:14.464485 9041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:14.496966 9041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:14.499560 9041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:14.499749 9041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:14.500056 9041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:28:14.500397 9041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:28:14.500432 9041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:14.500456 9041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:28:14.500499 9041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:14.507098 9041 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:41111
I20260812 05:28:14.507143 9173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:41111 every 8 connection(s)
I20260812 05:28:14.507483 9041 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:28:14.513264 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 9041
I20260812 05:28:14.513376 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:28:14.513571 24841 external_mini_cluster.cc:1428] Setting key 8a6e65ff935f0c0909aec48f7599676b
Aug 12 05:28:14 dist-test-slave-zpfg krb5kdc[8650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512494, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:28:14.519438 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.508289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:54855 (local address 127.24.66.126:35627)
0812 05:28:14.508368 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:14.508370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:14.508625 (+ 255us) server_negotiation.cc:407] Connection header received
0812 05:28:14.509425 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:14.509426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:14.509438 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:14.509487 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:14.510914 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.511451 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.513107 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.513420 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.513495 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.516315 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:14.516317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:14.516317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:14.516326 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:14.517960 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:14.518423 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:14.518425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:14.518426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:14.518485 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:14.518757 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:14.518757 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:14.518758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:14.518886 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:14.518962 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:14.519256 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 05:28:14.519346 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:28:14.519639 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.508259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:54855)
0812 05:28:14.508469 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:14.508523 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:14.509306 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:14.509586 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:14.509591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:14.510483 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:28:14.510800 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.510808 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.511551 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.511553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.513002 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.513006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.513520 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.513521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.513690 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.514444 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:28:14.514455 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:28:14.516207 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:28:14.518042 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:14.518051 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:14.518053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:14.518284 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:14.518563 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:14.518564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:14.518564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:14.518682 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:14.518964 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:28:14.518971 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:28:14.519117 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 05:28:14.519219 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 05:28:14.519392 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":109,"threads_started":1}
I20260812 05:28:14.520131 9174 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35627
I20260812 05:28:14.520216 9174 heartbeater.cc:464] Registering TS with master...
I20260812 05:28:14.520366 9174 heartbeater.cc:511] Master 127.24.66.126:35627 requested a full tablet report, sending...
I20260812 05:28:14.520699 8697 ts_manager.cc:194] Registered new tserver with Master: cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111)
I20260812 05:28:14.521108 8697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:54855
I20260812 05:28:14.526111 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:28:14.535331 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.527652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48658 (local address 127.24.66.126:35627)
0812 05:28:14.527702 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:14.527704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:14.527924 (+ 220us) server_negotiation.cc:407] Connection header received
0812 05:28:14.528066 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:14.528068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:14.528082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:14.528143 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:14.528934 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.529494 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.530490 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.530692 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.530753 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.532168 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:14.532169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:14.532170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:14.532176 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:14.533841 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:14.534432 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:14.534433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:14.534434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:14.534500 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:14.534771 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:14.534772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:14.534772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:14.534942 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:14.535036 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:14.535188 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 05:28:14.535261 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 05:28:14.537484 8697 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:28:14.539394 8697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:28:14.547297 9186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.542526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:41111 (local address 127.0.0.1:47524)
0812 05:28:14.542871 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:14.542920 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:14.543007 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:14.543310 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:14.543325 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:14.543327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:28:14.543410 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.543412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.544539 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.544541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.546116 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.546120 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.546978 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.546980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.547075 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.547089 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 05:28:14.547130 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 05:28:14.547204 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":214,"thread_start_us":67,"threads_started":1}
I20260812 05:28:14.547492 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.542526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:37919 (local address 127.0.0.1:34460)
0812 05:28:14.543189 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:14.543200 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:14.543258 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:14.543488 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:14.543489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:14.543492 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:28:14.543543 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.543544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.544889 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.544890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.546141 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.546145 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.547233 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.547234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.547293 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.547296 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:28:14.547362 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 05:28:14.547410 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":63,"thread_start_us":92,"threads_started":1}
I20260812 05:28:14.547765 9188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.542675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34460 (local address 127.24.66.66:37919)
0812 05:28:14.542882 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:14.542888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:14.543325 (+ 437us) server_negotiation.cc:407] Connection header received
0812 05:28:14.543372 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:14.543374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:14.543391 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:14.543472 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:28:14.543621 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.544771 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.546287 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.547137 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.547228 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.547299 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:14.547379 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 05:28:14.547432 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
I20260812 05:28:14.547886 9189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.542823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:34163 (local address 127.0.0.1:45846)
0812 05:28:14.543253 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:14.543263 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:14.543319 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:14.543581 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:14.543582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:14.543584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:28:14.543633 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.543635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.544918 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.544919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.546133 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.546134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.547661 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.547662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.547723 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.547727 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:28:14.547774 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 05:28:14.547812 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260812 05:28:14.547937 9187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.542638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47524 (local address 127.24.66.67:41111)
0812 05:28:14.543017 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:14.543021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:14.543035 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:28:14.543133 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:14.543136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:14.543165 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:14.543538 (+ 373us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:28:14.543623 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.544411 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.546235 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.546856 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.547601 (+ 745us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.547682 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:14.547709 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 05:28:14.547763 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":123,"threads_started":1}
I20260812 05:28:14.548187 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.543017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45846 (local address 127.24.66.65:34163)
0812 05:28:14.543205 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:14.543209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:14.543375 (+ 166us) server_negotiation.cc:407] Connection header received
0812 05:28:14.543421 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:14.543423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:14.543437 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:14.543502 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:28:14.543719 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.544816 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.546226 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.547137 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.547228 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.547299 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:14.548053 (+ 754us) server_negotiation.cc:299] Negotiation successful
0812 05:28:14.548108 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260812 05:28:14.548581 8968 tablet_service.cc:1467] Processing CreateTablet for tablet 8a1c195bbceb4be198138baa60f1f7c4 (DEFAULT_TABLE table=test-table [id=b56152e2968a40e3b076e4ba96a9f9a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:28:14.548590 9104 tablet_service.cc:1467] Processing CreateTablet for tablet 8a1c195bbceb4be198138baa60f1f7c4 (DEFAULT_TABLE table=test-table [id=b56152e2968a40e3b076e4ba96a9f9a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:28:14.548817 8828 tablet_service.cc:1467] Processing CreateTablet for tablet 8a1c195bbceb4be198138baa60f1f7c4 (DEFAULT_TABLE table=test-table [id=b56152e2968a40e3b076e4ba96a9f9a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:28:14.548993 8968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a1c195bbceb4be198138baa60f1f7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:28:14.548993 9104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a1c195bbceb4be198138baa60f1f7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:28:14.549170 8828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a1c195bbceb4be198138baa60f1f7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:28:14.551872 9191 tablet_bootstrap.cc:492] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Bootstrap starting.
I20260812 05:28:14.552815 9191 tablet_bootstrap.cc:654] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Neither blocks nor log segments found. Creating new log.
I20260812 05:28:14.552845 9192 tablet_bootstrap.cc:492] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: Bootstrap starting.
I20260812 05:28:14.553435 9193 tablet_bootstrap.cc:492] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: Bootstrap starting.
I20260812 05:28:14.553596 9192 tablet_bootstrap.cc:654] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: Neither blocks nor log segments found. Creating new log.
I20260812 05:28:14.553601 9191 log.cc:824] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Log is configured to *not* fsync() on all Append() calls
I20260812 05:28:14.554174 9192 log.cc:824] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: Log is configured to *not* fsync() on all Append() calls
I20260812 05:28:14.554116 9193 tablet_bootstrap.cc:654] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: Neither blocks nor log segments found. Creating new log.
I20260812 05:28:14.554652 9191 tablet_bootstrap.cc:492] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: No bootstrap required, opened a new log
I20260812 05:28:14.554749 9191 ts_tablet_manager.cc:1397] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:28:14.554929 9193 log.cc:824] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: Log is configured to *not* fsync() on all Append() calls
I20260812 05:28:14.555027 9192 tablet_bootstrap.cc:492] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: No bootstrap required, opened a new log
I20260812 05:28:14.555097 9192 ts_tablet_manager.cc:1397] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:28:14.556313 9191 raft_consensus.cc:348] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:14.556504 9191 raft_consensus.cc:374] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:28:14.556552 9191 raft_consensus.cc:729] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb207717cd0f40debf4ef346b3425be9, State: Initialized, Role: FOLLOWER
I20260812 05:28:14.556550 9192 raft_consensus.cc:348] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:14.556689 9191 consensus_queue.cc:260] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:14.556751 9192 raft_consensus.cc:374] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:28:14.556789 9192 raft_consensus.cc:729] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cce176cdb26c41ce977b9428c09d9335, State: Initialized, Role: FOLLOWER
I20260812 05:28:14.556897 9192 consensus_queue.cc:260] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:14.556998 9191 ts_tablet_manager.cc:1428] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:28:14.557150 8899 heartbeater.cc:503] Master 127.24.66.126:35627 was elected leader, sending a full tablet report...
I20260812 05:28:14.557448 9174 heartbeater.cc:503] Master 127.24.66.126:35627 was elected leader, sending a full tablet report...
I20260812 05:28:14.557780 9192 ts_tablet_manager.cc:1428] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:28:14.557814 9193 tablet_bootstrap.cc:492] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: No bootstrap required, opened a new log
I20260812 05:28:14.557884 9193 ts_tablet_manager.cc:1397] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 05:28:14.559525 9193 raft_consensus.cc:348] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:14.559716 9193 raft_consensus.cc:374] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:28:14.559768 9193 raft_consensus.cc:729] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c78683293248f08e8013fcaac49603, State: Initialized, Role: FOLLOWER
I20260812 05:28:14.559895 9193 consensus_queue.cc:260] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:14.560216 9193 ts_tablet_manager.cc:1428] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 05:28:14.560308 9036 heartbeater.cc:503] Master 127.24.66.126:35627 was elected leader, sending a full tablet report...
W20260812 05:28:14.563791 9037 tablet.cc:2369] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:28:14.637079 8900 tablet.cc:2369] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:28:14.758124 9175 tablet.cc:2369] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:28:14.767247 9199 raft_consensus.cc:482] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:28:14.767381 9199 raft_consensus.cc:504] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:14.767859 9199 leader_election.cc:288] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919), cb207717cd0f40debf4ef346b3425be9 (127.24.66.65:34163)
I20260812 05:28:14.771562 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.768515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:36331 (local address 127.24.66.65:34163)
0812 05:28:14.768583 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:14.768586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:14.768684 (+ 98us) server_negotiation.cc:407] Connection header received
0812 05:28:14.768775 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:14.768776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:14.768804 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:14.768867 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:28:14.769056 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.769534 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.770753 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.771138 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.771213 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.771223 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:14.771446 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 05:28:14.771495 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 05:28:14.771575 9201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.768423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:34163 (local address 127.24.66.67:36331)
0812 05:28:14.768601 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:14.768610 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:14.768698 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:14.768881 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:14.768882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:14.768888 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:28:14.768970 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.768976 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.769653 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.769654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.770658 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.770661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.771290 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.771291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.771364 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.771368 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:28:14.771440 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 05:28:14.771485 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":117,"threads_started":1}
I20260812 05:28:14.771660 9188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.768260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:33599 (local address 127.24.66.66:37919)
0812 05:28:14.768345 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:14.768348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:14.768367 (+ 19us) server_negotiation.cc:407] Connection header received
0812 05:28:14.768460 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:14.768462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:14.768476 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:14.768582 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:28:14.768816 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.769298 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.770854 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.771252 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.771353 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.771363 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:14.771560 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 05:28:14.771600 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 05:28:14.771658 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.768246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:37919 (local address 127.24.66.67:33599)
0812 05:28:14.768314 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:14.768327 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:14.768378 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:14.768626 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:14.768627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:14.768630 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:28:14.768704 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.768706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.769389 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.769390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.770643 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:14.770647 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.771379 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.771380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:14.771468 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.771471 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:28:14.771540 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 05:28:14.771586 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 05:28:14.771942 8988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a1c195bbceb4be198138baa60f1f7c4" candidate_uuid: "cce176cdb26c41ce977b9428c09d9335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c78683293248f08e8013fcaac49603" is_pre_election: true
I20260812 05:28:14.771926 8849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a1c195bbceb4be198138baa60f1f7c4" candidate_uuid: "cce176cdb26c41ce977b9428c09d9335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb207717cd0f40debf4ef346b3425be9" is_pre_election: true
I20260812 05:28:14.772104 8988 raft_consensus.cc:2454] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cce176cdb26c41ce977b9428c09d9335 in term 0.
I20260812 05:28:14.772105 8849 raft_consensus.cc:2454] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cce176cdb26c41ce977b9428c09d9335 in term 0.
I20260812 05:28:14.772384 9057 leader_election.cc:302] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c78683293248f08e8013fcaac49603, cce176cdb26c41ce977b9428c09d9335; no voters:
I20260812 05:28:14.772531 9199 raft_consensus.cc:2781] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:28:14.772600 9199 raft_consensus.cc:482] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:28:14.772637 9199 raft_consensus.cc:3037] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:28:14.773536 9199 raft_consensus.cc:504] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:14.773712 9199 leader_election.cc:288] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [CANDIDATE]: Term 1 election: Requested vote from peers 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919), cb207717cd0f40debf4ef346b3425be9 (127.24.66.65:34163)
I20260812 05:28:14.774015 8849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a1c195bbceb4be198138baa60f1f7c4" candidate_uuid: "cce176cdb26c41ce977b9428c09d9335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb207717cd0f40debf4ef346b3425be9"
I20260812 05:28:14.774130 8988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a1c195bbceb4be198138baa60f1f7c4" candidate_uuid: "cce176cdb26c41ce977b9428c09d9335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c78683293248f08e8013fcaac49603"
I20260812 05:28:14.774138 8849 raft_consensus.cc:3037] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:28:14.774216 8988 raft_consensus.cc:3037] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:28:14.775245 8988 raft_consensus.cc:2454] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cce176cdb26c41ce977b9428c09d9335 in term 1.
I20260812 05:28:14.775346 8849 raft_consensus.cc:2454] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cce176cdb26c41ce977b9428c09d9335 in term 1.
I20260812 05:28:14.775625 9057 leader_election.cc:302] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb207717cd0f40debf4ef346b3425be9, cce176cdb26c41ce977b9428c09d9335; no voters:
I20260812 05:28:14.775758 9199 raft_consensus.cc:2781] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:28:14.775967 9199 raft_consensus.cc:686] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 1 LEADER]: Becoming Leader. State: Replica: cce176cdb26c41ce977b9428c09d9335, State: Running, Role: LEADER
I20260812 05:28:14.776115 9199 consensus_queue.cc:237] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:14.777170 8697 catalog_manager.cc:5495] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 reported cstate change: term changed from 0 to 1, leader changed from <none> to cce176cdb26c41ce977b9428c09d9335 (127.24.66.67). New cstate: current_term: 1 leader_uuid: "cce176cdb26c41ce977b9428c09d9335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } health_report { overall_health: UNKNOWN } } }
I20260812 05:28:14.835536 9187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.832804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.24.66.67:41111)
0812 05:28:14.832894 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:14.832896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:14.832906 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:28:14.832945 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:14.832946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:14.832963 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:14.833041 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:28:14.833338 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.833838 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.834729 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:14.834920 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:14.834987 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:14.835141 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:28:14.835245 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:28:14.835307 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:14.835346 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 05:28:14.835365 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 05:28:14.837680 8849 raft_consensus.cc:1261] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Refusing update from remote peer cce176cdb26c41ce977b9428c09d9335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:28:14.837684 8988 raft_consensus.cc:1261] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603 [term 1 FOLLOWER]: Refusing update from remote peer cce176cdb26c41ce977b9428c09d9335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:28:14.838093 9199 consensus_queue.cc:1035] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:28:14.838415 9202 consensus_queue.cc:1035] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:28:14.841180 9206 mvcc.cc:205] Tried to move back new op lower bound from 7317555178851864576 to 7317555178603778048. Current Snapshot: MvccSnapshot[applied={T|T < 7317555178851864576}]
I20260812 05:28:14.843501 8697 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:28:14.843611 8697 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:28:14.844441 8697 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Sending DeleteTablet for 3 replicas of tablet 8a1c195bbceb4be198138baa60f1f7c4
I20260812 05:28:14.844838 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 8762
I20260812 05:28:14.844830 9104 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a1c195bbceb4be198138baa60f1f7c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:28:14 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:47524
I20260812 05:28:14.844905 8968 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a1c195bbceb4be198138baa60f1f7c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:28:14 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:34460
I20260812 05:28:14.844934 8828 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a1c195bbceb4be198138baa60f1f7c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:28:14 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:45846
I20260812 05:28:14.848309 9216 tablet_replica.cc:330] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: stopping tablet replica
I20260812 05:28:14.848416 9216 raft_consensus.cc:2229] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:28:14.848513 9216 raft_consensus.cc:2258] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:28:14.850800 9215 tablet_replica.cc:330] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: stopping tablet replica
I20260812 05:28:14.850970 9215 raft_consensus.cc:2229] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:28:14.851028 9216 ts_tablet_manager.cc:1905] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:28:14.851171 9215 raft_consensus.cc:2258] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:28:14.851599 9215 ts_tablet_manager.cc:1905] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 05:28:14.852898 8681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:34163 (error 108)
I20260812 05:28:14.853186 9216 ts_tablet_manager.cc:1918] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:28:14.853274 9216 log.cc:1196] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/8a1c195bbceb4be198138baa60f1f7c4
I20260812 05:28:14.853549 9216 ts_tablet_manager.cc:1939] T 8a1c195bbceb4be198138baa60f1f7c4 P 35c78683293248f08e8013fcaac49603: Deleting consensus metadata
I20260812 05:28:14.853706 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 8904
I20260812 05:28:14.855428 9215 ts_tablet_manager.cc:1918] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:28:14.855531 9215 log.cc:1196] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/8a1c195bbceb4be198138baa60f1f7c4
I20260812 05:28:14.855822 9215 ts_tablet_manager.cc:1939] T 8a1c195bbceb4be198138baa60f1f7c4 P cce176cdb26c41ce977b9428c09d9335: Deleting consensus metadata
I20260812 05:28:14.856272 8682 catalog_manager.cc:4841] TS cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111): tablet 8a1c195bbceb4be198138baa60f1f7c4 (table test-table [id=b56152e2968a40e3b076e4ba96a9f9a8]) successfully deleted
I20260812 05:28:14.857223 9189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.857072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:34163 (local address 127.0.0.1:45860)
0812 05:28:14.857141 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:14.857193 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:34163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 05:28:14.857475 8681 catalog_manager.cc:4584] TS cb207717cd0f40debf4ef346b3425be9 (127.24.66.65:34163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a1c195bbceb4be198138baa60f1f7c4: Network error: Client connection negotiation failed: client connection to 127.24.66.65:34163: connect: Connection refused (error 111)
I20260812 05:28:14.862448 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 9041
I20260812 05:28:14.865640 9189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:14.865466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:37919 (local address 127.0.0.1:34470)
0812 05:28:14.865553 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:14.865600 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.66:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 05:28:14.869967 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 8664
I20260812 05:28:34.877667 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35627
--webserver_interface=127.24.66.126
--webserver_port=37955
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:35627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:28:34.965679 9218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:34.965917 9218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:34.965942 9218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:34.967386 9218 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 05:28:34.967438 9218 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 05:28:34.967453 9218 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 05:28:34.967464 9218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:28:34.967479 9218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:34.967492 9218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:34.967545 9218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:28:34.967564 9218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:28:34.969157 9218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:35627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=37955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:34.969436 9218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:34.969717 9218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:28:34.979967 9225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:34.979982 9224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:34.980247 9218 server_base.cc:1034] running on GCE node
W20260812 05:28:34.979991 9227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:34.980676 9218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:34.981012 9218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:34.982191 9218 hybrid_clock.cc:648] HybridClock initialized: now 1786512514982177 us; error 35 us; skew 500 ppm
Aug 12 05:28:34 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512514, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:34.990270 9218 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:28:34.990823 9218 webserver.cc:460] Webserver started at http://127.24.66.126:37955/ using document root <none> and password file <none>
I20260812 05:28:34.991096 9218 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:34.991194 9218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:34.992806 9218 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:34.993633 9234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:34.993932 9218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:34.994071 9218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c"
format_stamp: "Formatted at 2026-08-12 05:28:13 on dist-test-slave-zpfg"
server_key: "53879f3fd81700ddc2456776b4b98b8f"
server_key_iv: "4b2146bae20660cdfb5ddcc5f108629a"
server_key_version: "encryptionkey@0"
I20260812 05:28:34.994607 9218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:35.003051 9218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:35.005667 9218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:35.005869 9218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:35.010686 9218 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:35627
I20260812 05:28:35.010720 9288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:35627 every 8 connection(s)
I20260812 05:28:35.011152 9218 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:28:35.012866 9289 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:28:35.013654 9289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Bootstrap starting.
I20260812 05:28:35.014683 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 9218
I20260812 05:28:35.015022 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:34163
--local_ip_for_outbound_sockets=127.24.66.65
--tserver_master_addrs=127.24.66.126:35627
--webserver_port=38659
--webserver_interface=127.24.66.65
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:28:35.024425 9289 log.cc:824] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:28:35.027311 9289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:28:35.027760 9289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Bootstrap complete.
I20260812 05:28:35.031038 9289 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:35.031363 9289 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8af8e0f532b4ed7bf2f9888c4b9547c, State: Initialized, Role: FOLLOWER
I20260812 05:28:35.031585 9289 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:35.031697 9289 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:28:35.031764 9289 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:28:35.031817 9289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 1 FOLLOWER]: Advancing to term 2
I20260812 05:28:35.032919 9289 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:35.033056 9289 leader_election.cc:302] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8af8e0f532b4ed7bf2f9888c4b9547c; no voters:
I20260812 05:28:35.033282 9289 leader_election.cc:288] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 05:28:35.033329 9293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 05:28:35.033454 9293 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 2 LEADER]: Becoming Leader. State: Replica: b8af8e0f532b4ed7bf2f9888c4b9547c, State: Running, Role: LEADER
I20260812 05:28:35.033577 9289 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:28:35.033629 9293 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:35.034065 9294 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } } }
I20260812 05:28:35.034171 9294 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: This master's current role is: LEADER
I20260812 05:28:35.034446 9295 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8af8e0f532b4ed7bf2f9888c4b9547c. Latest consensus state: current_term: 2 leader_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } } }
I20260812 05:28:35.034555 9295 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: This master's current role is: LEADER
I20260812 05:28:35.034626 9301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:28:35.035518 9301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:28:35.035696 9301 catalog_manager.cc:1259] Loaded cluster ID: 33d9025370634444ba0654666e7f3e87
I20260812 05:28:35.035945 9301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:28:35.037210 9301 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:28:35.037436 9301 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Loaded TSK: 0
I20260812 05:28:35.045939 9301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Generated new TSK 1
I20260812 05:28:35.046046 9301 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 05:28:35.106506 9291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:35.106889 9291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:35.106961 9291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:35.108916 9291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:35.109017 9291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:35.109091 9291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:28:35.111054 9291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:34163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=38659
--tserver_master_addrs=127.24.66.126:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:35.111328 9291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:35.111589 9291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:28:35.117007 9317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:35.117005 9316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:35.117190 9321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:35.118599 9291 server_base.cc:1034] running on GCE node
I20260812 05:28:35.118906 9291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:35.119221 9291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:35.120406 9291 hybrid_clock.cc:648] HybridClock initialized: now 1786512515120385 us; error 40 us; skew 500 ppm
Aug 12 05:28:35 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:35.123625 9291 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:28:35.124284 9291 webserver.cc:460] Webserver started at http://127.24.66.65:38659/ using document root <none> and password file <none>
I20260812 05:28:35.124567 9291 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:35.124636 9291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:35.126542 9291 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:28:35.127408 9328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:35.127627 9291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:35.127701 9291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "cb207717cd0f40debf4ef346b3425be9"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "bb4bec58c8167604d729b852f34c33a2"
server_key_iv: "e822680154559b72f79a869861c65ba5"
server_key_version: "encryptionkey@0"
I20260812 05:28:35.128118 9291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:35.171525 9291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:35.173862 9291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:35.174036 9291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:35.174371 9291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:28:35.174856 9335 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 05:28:35.176595 9291 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 05:28:35.176652 9291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 05:28:35.176678 9291 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 05:28:35.177426 9291 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 05:28:35.177461 9291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 05:28:35.177614 9335 tablet_bootstrap.cc:492] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Bootstrap starting.
I20260812 05:28:35.185277 9291 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:34163
I20260812 05:28:35.185389 9442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:34163 every 8 connection(s)
I20260812 05:28:35.185777 9291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:28:35.188344 9335 log.cc:824] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Log is configured to *not* fsync() on all Append() calls
I20260812 05:28:35.190896 9335 tablet_bootstrap.cc:492] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:28:35.191264 9335 tablet_bootstrap.cc:492] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Bootstrap complete.
I20260812 05:28:35.192380 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 9291
I20260812 05:28:35.192540 9335 ts_tablet_manager.cc:1397] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.011s
I20260812 05:28:35.192766 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:37919
--local_ip_for_outbound_sockets=127.24.66.66
--tserver_master_addrs=127.24.66.126:35627
--webserver_port=39079
--webserver_interface=127.24.66.66
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:28:35 dist-test-slave-zpfg krb5kdc[8650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:28:35.194236 9335 raft_consensus.cc:348] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:35.194644 9335 raft_consensus.cc:729] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb207717cd0f40debf4ef346b3425be9, State: Initialized, Role: FOLLOWER
I20260812 05:28:35.194901 9335 consensus_queue.cc:260] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:35.195272 9335 ts_tablet_manager.cc:1428] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 05:28:35.199090 9445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.186637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:40293)
0812 05:28:35.186849 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:35.186894 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:35.187429 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:35.187916 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:35.187920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:35.188214 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:28:35.188435 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.188442 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.189787 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.189789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.190741 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.190744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.191529 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.191531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.191669 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.192457 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:28:35.192481 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:28:35.194542 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:28:35.196867 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:35.196875 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:35.196878 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:35.197339 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:35.197708 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:35.197709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:35.197710 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:35.197875 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:35.198387 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:28:35.198395 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:28:35.198554 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 05:28:35.198686 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 05:28:35.198887 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":127,"threads_started":1}
I20260812 05:28:35.199090 9446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.186789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:40293 (local address 127.24.66.126:35627)
0812 05:28:35.187023 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.187032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.187075 (+ 43us) server_negotiation.cc:407] Connection header received
0812 05:28:35.187684 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.187688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.187735 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.187884 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:35.188607 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.189636 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.190842 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.191402 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.191537 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.194705 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:35.194711 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:35.194711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:35.194726 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:35.196759 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.197483 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.197486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.197487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.197593 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.197956 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.197957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.197957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.198288 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:35.198413 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:35.198757 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 05:28:35.198890 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":142,"threads_started":1}
I20260812 05:28:35.200052 9443 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35627
I20260812 05:28:35.200176 9443 heartbeater.cc:464] Registering TS with master...
I20260812 05:28:35.200415 9443 heartbeater.cc:511] Master 127.24.66.126:35627 requested a full tablet report, sending...
I20260812 05:28:35.201336 9252 ts_manager.cc:194] Registered new tserver with Master: cb207717cd0f40debf4ef346b3425be9 (127.24.66.65:34163)
I20260812 05:28:35.202615 9252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:40293
I20260812 05:28:35.203526 9443 heartbeater.cc:503] Master 127.24.66.126:35627 was elected leader, sending a full tablet report...
Aug 12 05:28:35 dist-test-slave-zpfg krb5kdc[8650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512514, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.65@KRBTEST.COM
I20260812 05:28:35.212103 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.202139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:34163 (local address 127.0.0.1:32884)
0812 05:28:35.202440 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:35.202481 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:35.202557 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:35.203146 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:35.203153 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:35.203612 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:28:35.203693 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.203696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.204722 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.204724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.205871 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.205875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.206402 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.206403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.206522 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.207029 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:28:35.207034 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:28:35.208709 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:28:35.210808 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:35.210816 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:35.210819 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:35.211037 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:35.211299 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:35.211300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:35.211301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:35.211371 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:35.211682 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:28:35.211690 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:28:35.211749 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 05:28:35.211856 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 05:28:35.212000 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":121,"threads_started":1}
I20260812 05:28:35.212103 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.202364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32884 (local address 127.24.66.65:34163)
0812 05:28:35.202943 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.202946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.202985 (+ 39us) server_negotiation.cc:407] Connection header received
0812 05:28:35.203021 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.203023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.203038 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.203131 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:35.203798 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.204629 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.205972 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.206283 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.206405 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.208863 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:35.208873 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:35.208874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:35.208886 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:35.210685 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.211145 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.211148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.211148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.211212 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.211440 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.211441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.211441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.211604 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:35.211706 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:35.211895 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 05:28:35.211998 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":80,"threads_started":1}
I20260812 05:28:35.212548 9377 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a1c195bbceb4be198138baa60f1f7c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:28:14 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:32884
I20260812 05:28:35.213130 9452 tablet_replica.cc:330] stopping tablet replica
I20260812 05:28:35.213268 9452 raft_consensus.cc:2229] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:28:35.213339 9452 raft_consensus.cc:2258] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:28:35.213698 9452 ts_tablet_manager.cc:1905] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:28:35.216492 9452 ts_tablet_manager.cc:1918] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:28:35.216588 9452 log.cc:1196] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal/wals/8a1c195bbceb4be198138baa60f1f7c4
I20260812 05:28:35.216869 9452 ts_tablet_manager.cc:1939] T 8a1c195bbceb4be198138baa60f1f7c4 P cb207717cd0f40debf4ef346b3425be9: Deleting consensus metadata
I20260812 05:28:35.217748 9235 catalog_manager.cc:4841] TS cb207717cd0f40debf4ef346b3425be9 (127.24.66.65:34163): tablet 8a1c195bbceb4be198138baa60f1f7c4 (table test-table [id=b56152e2968a40e3b076e4ba96a9f9a8]) successfully deleted
W20260812 05:28:35.289100 9448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:35.289330 9448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:35.289356 9448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:35.290673 9448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:35.290721 9448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:35.290755 9448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:28:35.292382 9448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:37919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=39079
--tserver_master_addrs=127.24.66.126:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:35.292706 9448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:35.292989 9448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:28:35.298611 9457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:35.298620 9458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:35.298597 9461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:35.298638 9448 server_base.cc:1034] running on GCE node
I20260812 05:28:35.299104 9448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:35.299456 9448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:35.300644 9448 hybrid_clock.cc:648] HybridClock initialized: now 1786512515300630 us; error 33 us; skew 500 ppm
Aug 12 05:28:35 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:35.303649 9448 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:28:35.304121 9448 webserver.cc:460] Webserver started at http://127.24.66.66:39079/ using document root <none> and password file <none>
I20260812 05:28:35.304401 9448 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:35.304493 9448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:35.306005 9448 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:35.306828 9468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:35.307020 9448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:28:35.307132 9448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "35c78683293248f08e8013fcaac49603"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "e015060a89cc03faae50091afd40a366"
server_key_iv: "cdb927752152df0a887112eb7165eb57"
server_key_version: "encryptionkey@0"
I20260812 05:28:35.307588 9448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:35.316133 9448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:35.318598 9448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:35.318764 9448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:35.319008 9448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:28:35.319336 9448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:28:35.319396 9448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:35.319440 9448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:28:35.319465 9448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:35.325841 9448 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:37919
I20260812 05:28:35.325886 9585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:37919 every 8 connection(s)
I20260812 05:28:35.326295 9448 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:28:35.330008 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 9448
I20260812 05:28:35.330422 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:41111
--local_ip_for_outbound_sockets=127.24.66.67
--tserver_master_addrs=127.24.66.126:35627
--webserver_port=37329
--webserver_interface=127.24.66.67
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 05:28:35 dist-test-slave-zpfg krb5kdc[8650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:28:35.337743 9446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.327081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:44953 (local address 127.24.66.126:35627)
0812 05:28:35.327146 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.327148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.327427 (+ 279us) server_negotiation.cc:407] Connection header received
0812 05:28:35.327996 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.327997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.328016 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.328067 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:35.329050 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.329569 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.331177 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.331464 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.331535 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.334493 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:35.334494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:35.334495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:35.334502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:35.335998 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.336579 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.336579 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.336580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.336633 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.336988 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.336989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.336989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.337116 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:35.337181 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:35.337584 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 05:28:35.337674 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 05:28:35.338069 9588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.327049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:44953)
0812 05:28:35.327272 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:35.327309 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:35.327868 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:35.328143 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:35.328147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:35.328729 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:28:35.328924 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.328929 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.329673 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.329674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.331037 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.331041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.331558 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.331559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.331717 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.332430 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:28:35.332439 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:28:35.334377 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:28:35.336122 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:35.336131 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:35.336134 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:35.336489 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:35.336713 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:35.336713 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:35.336714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:35.336875 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:35.337204 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:28:35.337211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:28:35.337407 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 05:28:35.337544 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 05:28:35.337760 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":100,"threads_started":1}
I20260812 05:28:35.338796 9586 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35627
I20260812 05:28:35.338892 9586 heartbeater.cc:464] Registering TS with master...
I20260812 05:28:35.339133 9586 heartbeater.cc:511] Master 127.24.66.126:35627 requested a full tablet report, sending...
I20260812 05:28:35.339673 9252 ts_manager.cc:194] Registered new tserver with Master: 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919)
I20260812 05:28:35.340119 9252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:44953
W20260812 05:28:35.427170 9589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:35.427462 9589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:35.427510 9589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:35.428915 9589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:35.429015 9589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:35.429065 9589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:28:35.430943 9589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:41111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=37329
--tserver_master_addrs=127.24.66.126:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:35.431258 9589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:35.431574 9589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:28:35.437575 9597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:35.437564 9594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:35.437703 9589 server_base.cc:1034] running on GCE node
W20260812 05:28:35.437559 9595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:35.438097 9589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:35.438405 9589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:35.439582 9589 hybrid_clock.cc:648] HybridClock initialized: now 1786512515439543 us; error 66 us; skew 500 ppm
Aug 12 05:28:35 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:35.442731 9589 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:28:35.443264 9589 webserver.cc:460] Webserver started at http://127.24.66.67:37329/ using document root <none> and password file <none>
I20260812 05:28:35.443477 9589 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:35.443538 9589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:35.445010 9589 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:35.445770 9604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:35.446013 9589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:35.446244 9589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "cce176cdb26c41ce977b9428c09d9335"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "a0444fd5b97526232384eea55fb34d41"
server_key_iv: "fb61ff593058d908757ca97a7b5ef5e8"
server_key_version: "encryptionkey@0"
I20260812 05:28:35.446689 9589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:35.480053 9589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:35.482697 9589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:35.482827 9589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:35.483173 9589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:28:35.483526 9589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:28:35.483585 9589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:35.483630 9589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:28:35.483657 9589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:35.490257 9589 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:41111
I20260812 05:28:35.490302 9721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:41111 every 8 connection(s)
I20260812 05:28:35.490717 9589 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
Aug 12 05:28:35 dist-test-slave-zpfg krb5kdc[8650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:28:35.498442 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 9589
I20260812 05:28:35.502025 9446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.491507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:52055 (local address 127.24.66.126:35627)
0812 05:28:35.491560 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.491563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.491774 (+ 211us) server_negotiation.cc:407] Connection header received
0812 05:28:35.492340 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.492341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.492351 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.492417 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:35.493573 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.494178 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.495370 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.495592 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.495660 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.498621 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:35.498623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:35.498623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:35.498632 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:35.500427 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.500914 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.500916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.500916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.500988 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.501271 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.501272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.501272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.501391 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:35.501494 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:35.501835 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 05:28:35.501925 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:28:35.502252 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.491453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:52055)
0812 05:28:35.491638 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:35.491683 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:35.492201 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:35.492501 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:35.492504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:35.493161 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:28:35.493370 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.493376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.494305 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.494306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.495257 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.495261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.495683 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.495684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.495826 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.496476 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:28:35.496485 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:28:35.498446 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:28:35.500546 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:35.500554 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:35.500556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:35.500809 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:35.501068 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:35.501069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:35.501069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:35.501192 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:35.501487 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:28:35.501494 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:28:35.501667 (+ 173us) client_negotiation.cc:769] Sending connection context
0812 05:28:35.501810 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 05:28:35.501992 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260812 05:28:35.502920 9723 heartbeater.cc:347] Connected to a master server at 127.24.66.126:35627
I20260812 05:28:35.503047 9723 heartbeater.cc:464] Registering TS with master...
I20260812 05:28:35.503304 9723 heartbeater.cc:511] Master 127.24.66.126:35627 requested a full tablet report, sending...
I20260812 05:28:35.503759 9252 ts_manager.cc:194] Registered new tserver with Master: cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111)
I20260812 05:28:35.504272 9252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:52055
I20260812 05:28:35.506546 9727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.499378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51238 (local address 127.24.66.126:35627)
0812 05:28:35.499505 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.499507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.499516 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:28:35.499543 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.499544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.499552 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.499640 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:35.500285 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.500817 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.501657 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.501889 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.501972 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.503633 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:35.503635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:35.503635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:35.503643 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:35.505249 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.505639 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.505640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.505640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.505699 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.505923 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.505924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.505924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.506047 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:35.506277 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:35.506318 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 05:28:35.506437 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260812 05:28:35.507313 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:28:35.510527 9727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.507669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.24.66.126:35627)
0812 05:28:35.507741 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.507743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.507896 (+ 153us) server_negotiation.cc:407] Connection header received
0812 05:28:35.507982 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.507983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.508022 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.508109 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:28:35.508325 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.508791 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.509664 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.509872 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.509964 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.510148 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:28:35.510199 (+ 51us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 05:28:35.510431 (+ 232us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51254: token expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 05:28:35.510617 9727 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51254: token expired
W20260812 05:28:35.510739 9180 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.66.126:35627)
I20260812 05:28:35.517184 9727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.511053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51262 (local address 127.24.66.126:35627)
0812 05:28:35.511097 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.511099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.511120 (+ 21us) server_negotiation.cc:407] Connection header received
0812 05:28:35.511136 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.511137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.511144 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.511177 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:35.511632 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.512158 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.512862 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.513045 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.513113 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.514299 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:35.514300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:35.514300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:35.514306 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:35.515955 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.516354 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.516355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.516356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.516419 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:35.516718 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:35.516719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:35.516719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:35.516834 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:35.516931 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:35.517029 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 05:28:35.517111 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:28:35.518811 9252 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:28:35.520581 9252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:28:35.523218 9377 tablet_service.cc:1467] Processing CreateTablet for tablet 77220db8b3164a0694c03a866d5f96fc (DEFAULT_TABLE table=test-table [id=6d14acdf5eb44a1389495c5e39484f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:28:35.523458 9377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77220db8b3164a0694c03a866d5f96fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:28:35.525134 9335 tablet_bootstrap.cc:492] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: Bootstrap starting.
I20260812 05:28:35.525461 9335 tablet_bootstrap.cc:654] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: Neither blocks nor log segments found. Creating new log.
I20260812 05:28:35.526559 9335 tablet_bootstrap.cc:492] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: No bootstrap required, opened a new log
I20260812 05:28:35.526621 9335 ts_tablet_manager.cc:1397] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:28:35.526788 9335 raft_consensus.cc:348] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:35.526873 9335 raft_consensus.cc:374] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:28:35.526922 9335 raft_consensus.cc:729] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb207717cd0f40debf4ef346b3425be9, State: Initialized, Role: FOLLOWER
I20260812 05:28:35.526973 9335 consensus_queue.cc:260] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:35.527073 9335 ts_tablet_manager.cc:1428] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:35.527158 9729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.522768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:37919 (local address 127.0.0.1:58072)
0812 05:28:35.522956 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:35.522964 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:35.522998 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:35.523618 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:35.523619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:35.523621 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:28:35.523669 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.523670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.524587 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.524588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.525789 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.525792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.526823 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.526825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.526895 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.526923 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 05:28:35.526994 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 05:28:35.527040 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":22,"thread_start_us":61,"threads_started":1}
I20260812 05:28:35.527302 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.522768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:41111 (local address 127.0.0.1:43534)
0812 05:28:35.522829 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:35.522841 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:35.522880 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:35.523597 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:35.523599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:35.523601 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:28:35.523654 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.523656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.524968 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.524970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.526001 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.526003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.527021 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.527022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.527102 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.527107 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:28:35.527166 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 05:28:35.527215 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 05:28:35.527487 9730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.522907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43534 (local address 127.24.66.67:41111)
0812 05:28:35.523384 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.523389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.523401 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:28:35.523442 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.523444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.523460 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.523748 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:28:35.523839 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.524863 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.526116 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.526933 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.527002 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.527057 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:35.527362 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 05:28:35.527415 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":105,"threads_started":1}
I20260812 05:28:35.527540 9731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.523141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.24.66.66:37919)
0812 05:28:35.523377 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.523382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.523395 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:28:35.523437 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.523439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.523456 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.523545 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:28:35.523791 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.524492 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.525891 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.526743 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.526821 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.526890 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:35.527404 (+ 514us) server_negotiation.cc:299] Negotiation successful
0812 05:28:35.527457 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":99,"threads_started":1}
I20260812 05:28:35.528232 9656 tablet_service.cc:1467] Processing CreateTablet for tablet 77220db8b3164a0694c03a866d5f96fc (DEFAULT_TABLE table=test-table [id=6d14acdf5eb44a1389495c5e39484f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:28:35.528258 9518 tablet_service.cc:1467] Processing CreateTablet for tablet 77220db8b3164a0694c03a866d5f96fc (DEFAULT_TABLE table=test-table [id=6d14acdf5eb44a1389495c5e39484f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:28:35.528621 9656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77220db8b3164a0694c03a866d5f96fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:28:35.528626 9518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77220db8b3164a0694c03a866d5f96fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:28:35.531649 9733 tablet_bootstrap.cc:492] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335: Bootstrap starting.
I20260812 05:28:35.531810 9734 tablet_bootstrap.cc:492] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: Bootstrap starting.
I20260812 05:28:35.532478 9733 tablet_bootstrap.cc:654] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335: Neither blocks nor log segments found. Creating new log.
I20260812 05:28:35.532480 9734 tablet_bootstrap.cc:654] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: Neither blocks nor log segments found. Creating new log.
I20260812 05:28:35.532964 9733 log.cc:824] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335: Log is configured to *not* fsync() on all Append() calls
I20260812 05:28:35.532989 9734 log.cc:824] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: Log is configured to *not* fsync() on all Append() calls
I20260812 05:28:35.533882 9734 tablet_bootstrap.cc:492] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: No bootstrap required, opened a new log
I20260812 05:28:35.533950 9733 tablet_bootstrap.cc:492] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335: No bootstrap required, opened a new log
I20260812 05:28:35.533962 9734 ts_tablet_manager.cc:1397] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:28:35.534015 9733 ts_tablet_manager.cc:1397] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:28:35.536067 9733 raft_consensus.cc:348] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:35.536067 9734 raft_consensus.cc:348] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:35.536295 9734 raft_consensus.cc:374] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:28:35.536294 9733 raft_consensus.cc:374] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:28:35.536336 9734 raft_consensus.cc:729] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c78683293248f08e8013fcaac49603, State: Initialized, Role: FOLLOWER
I20260812 05:28:35.536336 9733 raft_consensus.cc:729] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cce176cdb26c41ce977b9428c09d9335, State: Initialized, Role: FOLLOWER
I20260812 05:28:35.536473 9733 consensus_queue.cc:260] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:35.536473 9734 consensus_queue.cc:260] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:35.536780 9734 ts_tablet_manager.cc:1428] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:28:35.536782 9733 ts_tablet_manager.cc:1428] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 05:28:35.536811 9586 heartbeater.cc:503] Master 127.24.66.126:35627 was elected leader, sending a full tablet report...
I20260812 05:28:35.536921 9723 heartbeater.cc:503] Master 127.24.66.126:35627 was elected leader, sending a full tablet report...
W20260812 05:28:35.576953 9587 tablet.cc:2369] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:28:35.622642 9738 raft_consensus.cc:482] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:28:35.622843 9738 raft_consensus.cc:504] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:35.623350 9738 leader_election.cc:288] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919), cb207717cd0f40debf4ef346b3425be9 (127.24.66.65:34163)
I20260812 05:28:35.626673 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.623541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:37919 (local address 127.24.66.67:36229)
0812 05:28:35.623644 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:35.623661 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:35.623725 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:35.623880 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:35.623880 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:35.623882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:28:35.623955 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.623957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.624775 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.624776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.625599 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.625602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.626387 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.626388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.626475 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.626480 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:28:35.626547 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 05:28:35.626593 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 05:28:35.626803 9731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.623690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:36229 (local address 127.24.66.66:37919)
0812 05:28:35.623750 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.623752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.623775 (+ 23us) server_negotiation.cc:407] Connection header received
0812 05:28:35.623805 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.623806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.623820 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.623869 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:28:35.624058 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.624673 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.625715 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.626204 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.626263 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.626273 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:35.626685 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 05:28:35.626732 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:28:35.627059 9740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.623729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:34163 (local address 127.24.66.67:57523)
0812 05:28:35.623884 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:35.623899 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:35.623960 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:35.624142 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:35.624143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:35.624145 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:28:35.624212 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.624214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.625012 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.625013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.625846 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:35.625849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.626685 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.626686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:35.626797 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.626802 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:28:35.626941 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 05:28:35.626986 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260812 05:28:35.627071 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.623780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:57523 (local address 127.24.66.65:34163)
0812 05:28:35.623835 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.623837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.623981 (+ 144us) server_negotiation.cc:407] Connection header received
0812 05:28:35.624012 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.624013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.624039 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.624106 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:28:35.624304 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.624905 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.625944 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.626504 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.626571 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.626659 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:35.626958 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 05:28:35.627009 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:28:35.627136 9539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77220db8b3164a0694c03a866d5f96fc" candidate_uuid: "cce176cdb26c41ce977b9428c09d9335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c78683293248f08e8013fcaac49603" is_pre_election: true
I20260812 05:28:35.627380 9539 raft_consensus.cc:2454] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cce176cdb26c41ce977b9428c09d9335 in term 0.
I20260812 05:28:35.627422 9397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77220db8b3164a0694c03a866d5f96fc" candidate_uuid: "cce176cdb26c41ce977b9428c09d9335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb207717cd0f40debf4ef346b3425be9" is_pre_election: true
I20260812 05:28:35.627568 9397 raft_consensus.cc:2454] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cce176cdb26c41ce977b9428c09d9335 in term 0.
I20260812 05:28:35.627683 9605 leader_election.cc:302] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c78683293248f08e8013fcaac49603, cce176cdb26c41ce977b9428c09d9335; no voters:
I20260812 05:28:35.627887 9738 raft_consensus.cc:2781] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:28:35.627964 9738 raft_consensus.cc:482] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:28:35.627983 9738 raft_consensus.cc:3037] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:28:35.628796 9738 raft_consensus.cc:504] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:35.628945 9738 leader_election.cc:288] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [CANDIDATE]: Term 1 election: Requested vote from peers 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919), cb207717cd0f40debf4ef346b3425be9 (127.24.66.65:34163)
I20260812 05:28:35.629124 9539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77220db8b3164a0694c03a866d5f96fc" candidate_uuid: "cce176cdb26c41ce977b9428c09d9335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35c78683293248f08e8013fcaac49603"
I20260812 05:28:35.629137 9397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77220db8b3164a0694c03a866d5f96fc" candidate_uuid: "cce176cdb26c41ce977b9428c09d9335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb207717cd0f40debf4ef346b3425be9"
I20260812 05:28:35.629202 9539 raft_consensus.cc:3037] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:28:35.629310 9397 raft_consensus.cc:3037] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:28:35.629982 9539 raft_consensus.cc:2454] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cce176cdb26c41ce977b9428c09d9335 in term 1.
I20260812 05:28:35.630154 9397 raft_consensus.cc:2454] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cce176cdb26c41ce977b9428c09d9335 in term 1.
I20260812 05:28:35.630163 9605 leader_election.cc:302] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c78683293248f08e8013fcaac49603, cce176cdb26c41ce977b9428c09d9335; no voters:
I20260812 05:28:35.630276 9738 raft_consensus.cc:2781] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:28:35.630481 9738 raft_consensus.cc:686] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 1 LEADER]: Becoming Leader. State: Replica: cce176cdb26c41ce977b9428c09d9335, State: Running, Role: LEADER
I20260812 05:28:35.630611 9738 consensus_queue.cc:237] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:35.631611 9251 catalog_manager.cc:5495] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 reported cstate change: term changed from 0 to 1, leader changed from <none> to cce176cdb26c41ce977b9428c09d9335 (127.24.66.67). New cstate: current_term: 1 leader_uuid: "cce176cdb26c41ce977b9428c09d9335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } health_report { overall_health: UNKNOWN } } }
I20260812 05:28:35.647905 9730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.645352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43536 (local address 127.24.66.67:41111)
0812 05:28:35.645427 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:35.645429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:35.645448 (+ 19us) server_negotiation.cc:407] Connection header received
0812 05:28:35.645500 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:35.645501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:35.645519 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:35.645600 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:28:35.645780 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.646270 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.647051 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:35.647255 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:35.647340 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:35.647524 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:28:35.647630 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:28:35.647744 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:35.647762 (+ 18us) server_negotiation.cc:299] Negotiation successful
0812 05:28:35.647786 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:28:35.650046 9539 raft_consensus.cc:1261] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603 [term 1 FOLLOWER]: Refusing update from remote peer cce176cdb26c41ce977b9428c09d9335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:28:35.650074 9397 raft_consensus.cc:1261] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Refusing update from remote peer cce176cdb26c41ce977b9428c09d9335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:28:35.650669 9741 consensus_queue.cc:1035] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:28:35.650838 9738 consensus_queue.cc:1035] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:28:35.653472 9744 mvcc.cc:205] Tried to move back new op lower bound from 7317555264099487744 to 7317555264023904256. Current Snapshot: MvccSnapshot[applied={T|T < 7317555264099487744}]
I20260812 05:28:35.653927 9745 mvcc.cc:205] Tried to move back new op lower bound from 7317555264099487744 to 7317555264023904256. Current Snapshot: MvccSnapshot[applied={T|T < 7317555264099487744}]
I20260812 05:28:35.656194 9252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:28:35.656304 9252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:28:35.657155 9252 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Sending DeleteTablet for 3 replicas of tablet 77220db8b3164a0694c03a866d5f96fc
I20260812 05:28:35.657560 9518 tablet_service.cc:1514] Processing DeleteTablet for tablet 77220db8b3164a0694c03a866d5f96fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:28:35 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:58072
I20260812 05:28:35.657564 9656 tablet_service.cc:1514] Processing DeleteTablet for tablet 77220db8b3164a0694c03a866d5f96fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:28:35 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:43534
I20260812 05:28:35.657811 9377 tablet_service.cc:1514] Processing DeleteTablet for tablet 77220db8b3164a0694c03a866d5f96fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:28:35 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:32884
I20260812 05:28:35.657829 9752 tablet_replica.cc:330] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335: stopping tablet replica
I20260812 05:28:35.657912 9452 tablet_replica.cc:330] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: stopping tablet replica
I20260812 05:28:35.657934 9752 raft_consensus.cc:2229] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:28:35.657966 9452 raft_consensus.cc:2229] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:28:35.658018 9452 raft_consensus.cc:2258] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:28:35.658071 9752 raft_consensus.cc:2258] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:28:35.658165 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 9291
I20260812 05:28:35.658578 9752 ts_tablet_manager.cc:1905] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:28:35.660133 9754 tablet_replica.cc:330] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: stopping tablet replica
I20260812 05:28:35.660262 9754 raft_consensus.cc:2229] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:28:35.660383 9754 raft_consensus.cc:2258] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:28:35.660574 9752 ts_tablet_manager.cc:1918] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:28:35.660660 9752 log.cc:1196] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/77220db8b3164a0694c03a866d5f96fc
I20260812 05:28:35.660981 9752 ts_tablet_manager.cc:1939] T 77220db8b3164a0694c03a866d5f96fc P cce176cdb26c41ce977b9428c09d9335: Deleting consensus metadata
I20260812 05:28:35.661458 9236 catalog_manager.cc:4841] TS cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111): tablet 77220db8b3164a0694c03a866d5f96fc (table test-table [id=6d14acdf5eb44a1389495c5e39484f3c]) successfully deleted
I20260812 05:28:35.661960 9754 ts_tablet_manager.cc:1905] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:28:35.663717 9754 ts_tablet_manager.cc:1918] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:28:35.663816 9754 log.cc:1196] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/77220db8b3164a0694c03a866d5f96fc
I20260812 05:28:35.664136 9754 ts_tablet_manager.cc:1939] T 77220db8b3164a0694c03a866d5f96fc P 35c78683293248f08e8013fcaac49603: Deleting consensus metadata
I20260812 05:28:35.664611 9235 catalog_manager.cc:4841] TS 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919): tablet 77220db8b3164a0694c03a866d5f96fc (table test-table [id=6d14acdf5eb44a1389495c5e39484f3c]) successfully deleted
W20260812 05:28:35.667438 9235 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:34163 (error 108)
I20260812 05:28:35.667889 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 9448
I20260812 05:28:35.669075 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:35.668022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:34163 (local address 127.0.0.1:32900)
0812 05:28:35.668968 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:35.669022 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:34163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":917}
W20260812 05:28:35.670629 9235 catalog_manager.cc:4584] TS cb207717cd0f40debf4ef346b3425be9 (127.24.66.65:34163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77220db8b3164a0694c03a866d5f96fc: Network error: Client connection negotiation failed: client connection to 127.24.66.65:34163: connect: Connection refused (error 111)
I20260812 05:28:35.676128 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 9589
I20260812 05:28:35.683108 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 9218
I20260812 05:28:55.690616 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35627
--webserver_interface=127.24.66.126
--webserver_port=37955
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:35627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:28:55.777666 9756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:55.777927 9756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:55.777966 9756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:55.779224 9756 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 05:28:55.779284 9756 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 05:28:55.779306 9756 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 05:28:55.779326 9756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:28:55.779358 9756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:55.779385 9756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:55.779407 9756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:28:55.779434 9756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:28:55.780951 9756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:35627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=37955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:55.781241 9756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:55.781481 9756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:28:55.787177 9761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:55.787261 9762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:55.787177 9765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:55.787478 9756 server_base.cc:1034] running on GCE node
I20260812 05:28:55.787753 9756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:55.788071 9756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:55.789268 9756 hybrid_clock.cc:648] HybridClock initialized: now 1786512535789258 us; error 37 us; skew 500 ppm
Aug 12 05:28:55 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512535, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:55.798254 9756 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:28:55.798821 9756 webserver.cc:460] Webserver started at http://127.24.66.126:37955/ using document root <none> and password file <none>
I20260812 05:28:55.799033 9756 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:55.799078 9756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:55.800488 9756 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:55.801324 9773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:55.801546 9756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:55.801649 9756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c"
format_stamp: "Formatted at 2026-08-12 05:28:13 on dist-test-slave-zpfg"
server_key: "53879f3fd81700ddc2456776b4b98b8f"
server_key_iv: "4b2146bae20660cdfb5ddcc5f108629a"
server_key_version: "encryptionkey@0"
I20260812 05:28:55.801999 9756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:55.812333 9756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:55.814695 9756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:55.814819 9756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:55.819141 9756 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:35627
I20260812 05:28:55.819195 9826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:35627 every 8 connection(s)
I20260812 05:28:55.819586 9756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:28:55.821214 9827 sys_catalog.cc:263] Verifying existing consensus state
I20260812 05:28:55.821837 9827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Bootstrap starting.
I20260812 05:28:55.827440 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 9756
I20260812 05:28:55.827865 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:34163
--local_ip_for_outbound_sockets=127.24.66.65
--tserver_master_addrs=127.24.66.126:35627
--webserver_port=38659
--webserver_interface=127.24.66.65
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:28:55.832526 9827 log.cc:824] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:28:55.835119 9827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:28:55.835465 9827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Bootstrap complete.
I20260812 05:28:55.837853 9827 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:55.838126 9827 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8af8e0f532b4ed7bf2f9888c4b9547c, State: Initialized, Role: FOLLOWER
I20260812 05:28:55.838362 9827 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:55.838462 9827 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:28:55.838524 9827 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:28:55.838575 9827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 2 FOLLOWER]: Advancing to term 3
I20260812 05:28:55.839664 9827 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:55.839793 9827 leader_election.cc:302] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8af8e0f532b4ed7bf2f9888c4b9547c; no voters:
I20260812 05:28:55.840001 9827 leader_election.cc:288] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 05:28:55.840083 9831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 05:28:55.840243 9831 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [term 3 LEADER]: Becoming Leader. State: Replica: b8af8e0f532b4ed7bf2f9888c4b9547c, State: Running, Role: LEADER
I20260812 05:28:55.840292 9827 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:28:55.840359 9831 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } }
I20260812 05:28:55.840626 9832 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } } }
I20260812 05:28:55.840716 9832 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: This master's current role is: LEADER
I20260812 05:28:55.840623 9831 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8af8e0f532b4ed7bf2f9888c4b9547c. Latest consensus state: current_term: 3 leader_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8af8e0f532b4ed7bf2f9888c4b9547c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35627 } } }
I20260812 05:28:55.840889 9831 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c [sys.catalog]: This master's current role is: LEADER
I20260812 05:28:55.840993 9836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:28:55.841809 9836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:28:55.841917 9836 catalog_manager.cc:1259] Loaded cluster ID: 33d9025370634444ba0654666e7f3e87
I20260812 05:28:55.841949 9836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:28:55.843242 9836 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:28:55.843437 9836 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Loaded TSK: 0
I20260812 05:28:55.843482 9836 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b8af8e0f532b4ed7bf2f9888c4b9547c: Loaded TSK: 1
F20260812 05:28:55.843705 9836 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa7be4eb9ad google::LogMessage::Fail() at ??:0
@ 0x7fa7be4ee88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa7be4eb4de google::LogMessage::Flush() at ??:0
@ 0x7fa7be4eca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa7c0c58104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa7c0c55d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa7c64451cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa7c644548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa7c6445594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fa7c640bc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa7c6415ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa7c03b90d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa7c03ae954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa7c12126db start_thread at ??:0
@ 0x7fa7bcfe961f clone at ??:0
W20260812 05:28:55.939036 9829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:55.939265 9829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:55.939291 9829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:55.940493 9829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:55.940538 9829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:55.940565 9829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:28:55.942072 9829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:34163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=38659
--tserver_master_addrs=127.24.66.126:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:55.942350 9829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:55.942627 9829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:28:55.949359 9870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:55.949369 9869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:55.949666 9829 server_base.cc:1034] running on GCE node
W20260812 05:28:55.949841 9872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:55.950129 9829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:55.950446 9829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:55.951627 9829 hybrid_clock.cc:648] HybridClock initialized: now 1786512535951607 us; error 48 us; skew 500 ppm
Aug 12 05:28:55 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512535, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:55.954586 9829 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:28:55.955127 9829 webserver.cc:460] Webserver started at http://127.24.66.65:38659/ using document root <none> and password file <none>
I20260812 05:28:55.955377 9829 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:55.955447 9829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:55.956948 9829 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:28:55.957863 9879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:55.958047 9829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:55.958233 9829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "cb207717cd0f40debf4ef346b3425be9"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "bb4bec58c8167604d729b852f34c33a2"
server_key_iv: "e822680154559b72f79a869861c65ba5"
server_key_version: "encryptionkey@0"
I20260812 05:28:55.958668 9829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:55.968593 9829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:55.971345 9829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:55.971520 9829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:55.971767 9829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:28:55.972259 9887 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 05:28:55.974682 9829 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 05:28:55.974776 9829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260812 05:28:55.974833 9829 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 05:28:55.975536 9829 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 05:28:55.975592 9829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 05:28:55.975684 9887 tablet_bootstrap.cc:492] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: Bootstrap starting.
I20260812 05:28:55.983407 9829 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:34163
I20260812 05:28:55.983580 10000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:34163 every 8 connection(s)
I20260812 05:28:55.983870 9829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:28:55.985018 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 9829
I20260812 05:28:55.985356 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:37919
--local_ip_for_outbound_sockets=127.24.66.66
--tserver_master_addrs=127.24.66.126:35627
--webserver_port=39079
--webserver_interface=127.24.66.66
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:28:55.988129 9887 log.cc:824] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: Log is configured to *not* fsync() on all Append() calls
I20260812 05:28:55.991520 9887 tablet_bootstrap.cc:492] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 05:28:55.991945 9887 tablet_bootstrap.cc:492] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: Bootstrap complete.
I20260812 05:28:55.993378 9887 ts_tablet_manager.cc:1397] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.005s
I20260812 05:28:55.995107 9887 raft_consensus.cc:348] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:55.995543 9887 raft_consensus.cc:729] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb207717cd0f40debf4ef346b3425be9, State: Initialized, Role: FOLLOWER
I20260812 05:28:55.995740 9887 consensus_queue.cc:260] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:55.996073 9887 ts_tablet_manager.cc:1428] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9: Time spent starting tablet: real 0.003s user 0.003s sys 0.002s
W20260812 05:28:56.070034 10003 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:28:55.984602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:56351)
0812 05:28:55.984781 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:55.984827 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:55.985561 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:56.069804 (+ 84243us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
W20260812 05:28:56.070463 10001 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:35627 (0 consecutive failures): Network error: Failed to ping master at 127.24.66.126:35627: Client connection negotiation failed: client connection to 127.24.66.126:35627: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 05:28:56.070873 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:56.070728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:34337)
0812 05:28:56.070795 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:56.070844 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 05:28:56.071317 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:56.071186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:45153)
0812 05:28:56.071257 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:56.071292 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 05:28:56.071482 10001 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:28:56.075285 10004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:56.075510 10004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:56.075536 10004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:56.076730 10004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:56.076773 10004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:56.076800 10004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:28:56.078250 10004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:37919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=39079
--tserver_master_addrs=127.24.66.126:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:56.078516 10004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:56.078751 10004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:28:56.084136 10013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:56.084182 10004 server_base.cc:1034] running on GCE node
W20260812 05:28:56.084136 10012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:56.084187 10015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:56.084663 10004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:56.084934 10004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:56.086107 10004 hybrid_clock.cc:648] HybridClock initialized: now 1786512536086088 us; error 61 us; skew 500 ppm
Aug 12 05:28:56 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512536, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:56.088910 10004 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:28:56.089404 10004 webserver.cc:460] Webserver started at http://127.24.66.66:39079/ using document root <none> and password file <none>
I20260812 05:28:56.089637 10004 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:56.089706 10004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:56.091307 10004 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:56.092027 10022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:56.092214 10004 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:28:56.092344 10004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "35c78683293248f08e8013fcaac49603"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "e015060a89cc03faae50091afd40a366"
server_key_iv: "cdb927752152df0a887112eb7165eb57"
server_key_version: "encryptionkey@0"
I20260812 05:28:56.092700 10004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:56.111006 10004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:56.113148 10004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:56.113322 10004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:56.113603 10004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:28:56.113920 10004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:28:56.113974 10004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:56.114015 10004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:28:56.114041 10004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:56.120589 10004 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:37919
I20260812 05:28:56.120646 10139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:37919 every 8 connection(s)
I20260812 05:28:56.120993 10004 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:28:56.121857 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 10004
I20260812 05:28:56.122224 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:41111
--local_ip_for_outbound_sockets=127.24.66.67
--tserver_master_addrs=127.24.66.126:35627
--webserver_port=37329
--webserver_interface=127.24.66.67
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:28:56.122291 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:56.121782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:53649)
0812 05:28:56.121988 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:56.122090 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":76,"threads_started":1}
W20260812 05:28:56.122776 10140 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:35627 (0 consecutive failures): Network error: Failed to ping master at 127.24.66.126:35627: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
I20260812 05:28:56.123185 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:56.122976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:58527)
0812 05:28:56.123114 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:56.123147 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260812 05:28:56.123530 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:56.123444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:48867)
0812 05:28:56.123485 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:56.123513 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 05:28:56.123628 10140 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 05:28:56.209595 10144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:28:56.209856 10144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:28:56.209903 10144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:28:56.211186 10144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:28:56.211257 10144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:28:56.211302 10144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:28:56.212814 10144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:41111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=37329
--tserver_master_addrs=127.24.66.126:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:28:56.213091 10144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:28:56.213557 10144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:28:56.219096 10144 server_base.cc:1034] running on GCE node
W20260812 05:28:56.219246 10150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:56.219194 10152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:28:56.219090 10149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:28:56.219704 10144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:28:56.220032 10144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:28:56.221207 10144 hybrid_clock.cc:648] HybridClock initialized: now 1786512536221196 us; error 30 us; skew 500 ppm
Aug 12 05:28:56 dist-test-slave-zpfg krb5kdc[8650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512536, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:28:56.223837 10144 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:28:56.224273 10144 webserver.cc:460] Webserver started at http://127.24.66.67:37329/ using document root <none> and password file <none>
I20260812 05:28:56.224522 10144 fs_manager.cc:359] Metadata directory not provided
I20260812 05:28:56.224570 10144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:28:56.225977 10144 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:28:56.226696 10160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:56.226986 10144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:28:56.227084 10144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "cce176cdb26c41ce977b9428c09d9335"
format_stamp: "Formatted at 2026-08-12 05:28:14 on dist-test-slave-zpfg"
server_key: "a0444fd5b97526232384eea55fb34d41"
server_key_iv: "fb61ff593058d908757ca97a7b5ef5e8"
server_key_version: "encryptionkey@0"
I20260812 05:28:56.227569 10144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:28:56.249033 10144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:28:56.251322 10144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:28:56.251484 10144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:28:56.251773 10144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:28:56.252105 10144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:28:56.252139 10144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:56.252161 10144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:28:56.252205 10144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:28:56.258729 10144 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:41111
I20260812 05:28:56.258788 10275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:41111 every 8 connection(s)
I20260812 05:28:56.259142 10144 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:28:56.260246 10278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:56.259858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:48069)
0812 05:28:56.260022 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:56.260098 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
W20260812 05:28:56.260700 10276 heartbeater.cc:644] Failed to heartbeat to 127.24.66.126:35627 (0 consecutive failures): Network error: Failed to ping master at 127.24.66.126:35627: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
I20260812 05:28:56.261020 10278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:56.260932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:51701)
0812 05:28:56.260976 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:56.261003 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 05:28:56.261245 10278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:56.261196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:58037)
0812 05:28:56.261216 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:56.261234 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260812 05:28:56.261400 10276 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 05:28:56.268886 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 10144
I20260812 05:28:57.072194 10280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:57.071881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:36011)
0812 05:28:57.072036 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:57.072081 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260812 05:28:57.124323 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:57.124032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:58447)
0812 05:28:57.124202 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:57.124249 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260812 05:28:57.261961 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:57.261718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:60943)
0812 05:28:57.261869 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:57.261908 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260812 05:28:57.714648 10283 raft_consensus.cc:482] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:28:57.714819 10283 raft_consensus.cc:504] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:57.715250 10283 leader_election.cc:288] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919), cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111)
Aug 12 05:28:57 dist-test-slave-zpfg krb5kdc[8650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512535, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.66@KRBTEST.COM
Aug 12 05:28:57 dist-test-slave-zpfg krb5kdc[8650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512535, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.67@KRBTEST.COM
I20260812 05:28:57.726823 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:57.715519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:41251 (local address 127.24.66.66:37919)
0812 05:28:57.715668 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:57.715681 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:57.715707 (+ 26us) server_negotiation.cc:407] Connection header received
0812 05:28:57.716189 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:57.716192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:57.716235 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:57.716388 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:57.717149 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:57.718041 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:57.719151 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:57.719590 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:57.719850 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:57.722154 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:57.722159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:57.722160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:57.722173 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:57.724721 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:57.725444 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:57.725448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:57.725448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:57.725541 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:57.725839 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:57.725840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:57.725841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:57.726092 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:57.726199 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:57.726528 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 05:28:57.726702 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":44,"threads_started":1}
I20260812 05:28:57.726830 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:57.715400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:37919 (local address 127.24.66.65:41251)
0812 05:28:57.715585 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:57.715596 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:57.715660 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:57.716430 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:57.716433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:57.716827 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:28:57.717026 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:57.717032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:57.718150 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:57.718151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:57.719046 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:57.719049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:57.719803 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:57.719804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:57.719883 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:57.720412 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:28:57.720424 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:28:57.721998 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:28:57.724805 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:57.724809 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:57.724812 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:57.725345 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:57.725629 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:57.725630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:57.725630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:57.725759 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:57.726178 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:28:57.726185 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:28:57.726320 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 05:28:57.726472 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 05:28:57.726733 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":123,"thread_start_us":63,"threads_started":1}
I20260812 05:28:57.726907 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:57.715539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:42157 (local address 127.24.66.67:41111)
0812 05:28:57.715725 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 05:28:57.715737 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:28:57.715790 (+ 53us) server_negotiation.cc:407] Connection header received
0812 05:28:57.716289 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:28:57.716292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:28:57.716333 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:28:57.716483 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:28:57.717198 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:57.718124 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:57.719144 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:28:57.719590 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:28:57.719725 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:57.722495 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:28:57.722498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:28:57.722499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:28:57.722512 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:28:57.724651 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:57.725447 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:57.725455 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:57.725455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:57.725552 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:28:57.725852 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:28:57.725853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:28:57.725854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:28:57.726109 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:28:57.726210 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 05:28:57.726562 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 05:28:57.726720 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260812 05:28:57.727002 10285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:57.715431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:41111 (local address 127.24.66.65:42157)
0812 05:28:57.715638 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:57.715652 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:28:57.715721 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:28:57.716485 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:28:57.716486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:28:57.716828 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:28:57.717032 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:57.717034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:57.718229 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:57.718230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:57.719048 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:28:57.719050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:28:57.719695 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:28:57.719696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:28:57.719836 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:28:57.720562 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:28:57.720567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:28:57.722388 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:28:57.724798 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:57.724809 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:57.724812 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:57.725300 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:57.725645 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:28:57.725646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:28:57.725646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:28:57.725765 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:28:57.726189 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:28:57.726190 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:28:57.726321 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 05:28:57.726487 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 05:28:57.726825 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":198,"thread_start_us":90,"threads_started":1}
W20260812 05:28:57.728360 9880 leader_election.cc:341] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919): Not found: Tablet not found: 77220db8b3164a0694c03a866d5f96fc
W20260812 05:28:57.728583 9881 leader_election.cc:341] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111): Not found: Tablet not found: 77220db8b3164a0694c03a866d5f96fc
I20260812 05:28:57.728652 9881 leader_election.cc:302] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb207717cd0f40debf4ef346b3425be9; no voters: 35c78683293248f08e8013fcaac49603, cce176cdb26c41ce977b9428c09d9335
I20260812 05:28:57.728809 10283 raft_consensus.cc:2726] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:28:58.072983 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:58.072800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:52013)
0812 05:28:58.072887 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:58.072938 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 05:28:58.125119 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:58.124882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:56005)
0812 05:28:58.125014 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:58.125062 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260812 05:28:58.262951 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:58.262713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:35033)
0812 05:28:58.262858 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:58.262904 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260812 05:28:59.073864 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:59.073585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:55653)
0812 05:28:59.073755 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:59.073814 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260812 05:28:59.125895 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:59.125624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:49421)
0812 05:28:59.125804 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:59.125848 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
I20260812 05:28:59.263749 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:28:59.263500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:60249)
0812 05:28:59.263646 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 05:28:59.263697 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260812 05:28:59.718518 10293 raft_consensus.cc:482] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:28:59.718621 10293 raft_consensus.cc:504] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:28:59.718822 10293 leader_election.cc:288] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919), cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111)
W20260812 05:28:59.719360 9881 leader_election.cc:341] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111): Not found: Tablet not found: 77220db8b3164a0694c03a866d5f96fc
W20260812 05:28:59.719477 9880 leader_election.cc:341] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919): Not found: Tablet not found: 77220db8b3164a0694c03a866d5f96fc
I20260812 05:28:59.719563 9880 leader_election.cc:302] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb207717cd0f40debf4ef346b3425be9; no voters: 35c78683293248f08e8013fcaac49603, cce176cdb26c41ce977b9428c09d9335
I20260812 05:28:59.719662 10293 raft_consensus.cc:2726] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:29:00.074888 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:00.074606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:35991)
0812 05:29:00.074762 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:00.074814 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260812 05:29:00.126915 10295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:00.126592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:34503)
0812 05:29:00.126797 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:00.126845 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":69,"threads_started":1}
I20260812 05:29:00.264616 10296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:00.264376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:43659)
0812 05:29:00.264518 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:00.264565 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260812 05:29:01.075860 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:01.075535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:60289)
0812 05:29:01.075734 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:01.075785 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":77,"threads_started":1}
I20260812 05:29:01.127993 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:01.127618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:35453)
0812 05:29:01.127894 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:01.127941 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":156,"threads_started":1}
I20260812 05:29:01.265503 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:01.265261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:56629)
0812 05:29:01.265411 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:01.265455 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260812 05:29:01.327344 10300 raft_consensus.cc:482] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:29:01.327521 10300 raft_consensus.cc:504] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:29:01.327823 10300 leader_election.cc:288] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919), cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111)
W20260812 05:29:01.328322 9880 leader_election.cc:341] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919): Not found: Tablet not found: 77220db8b3164a0694c03a866d5f96fc
W20260812 05:29:01.328440 9881 leader_election.cc:341] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111): Not found: Tablet not found: 77220db8b3164a0694c03a866d5f96fc
I20260812 05:29:01.328517 9881 leader_election.cc:302] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb207717cd0f40debf4ef346b3425be9; no voters: 35c78683293248f08e8013fcaac49603, cce176cdb26c41ce977b9428c09d9335
I20260812 05:29:01.328652 10300 raft_consensus.cc:2726] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:29:02.076822 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:02.076543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:49511)
0812 05:29:02.076717 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:02.076769 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":74,"threads_started":1}
I20260812 05:29:02.128983 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:02.128711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:55681)
0812 05:29:02.128882 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:02.128930 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":73,"threads_started":1}
I20260812 05:29:02.266477 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:02.266034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:55917)
0812 05:29:02.266243 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:02.266422 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":67,"threads_started":1}
I20260812 05:29:03.077818 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:03.077506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:35455)
0812 05:29:03.077702 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:03.077755 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
I20260812 05:29:03.129941 10305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:03.129664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:57293)
0812 05:29:03.129822 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:03.129890 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":66,"threads_started":1}
I20260812 05:29:03.267341 10306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:03.267056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:47837)
0812 05:29:03.267234 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:03.267281 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260812 05:29:04.078958 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:04.078690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:54415)
0812 05:29:04.078848 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:04.078908 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260812 05:29:04.130908 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:04.130525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:38447)
0812 05:29:04.130761 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:04.130855 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260812 05:29:04.268318 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:04.267965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:56535)
0812 05:29:04.268211 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:04.268265 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":94,"threads_started":1}
I20260812 05:29:05.079785 10310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:05.079497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:32879)
0812 05:29:05.079671 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:05.079731 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260812 05:29:05.131870 10311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:05.131513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:40801)
0812 05:29:05.131751 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:05.131812 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":66,"threads_started":1}
I20260812 05:29:05.269218 10312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:05.268943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:49667)
0812 05:29:05.269123 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:05.269168 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
I20260812 05:29:05.581743 10313 raft_consensus.cc:482] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:29:05.581856 10313 raft_consensus.cc:504] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:29:05.582085 10313 leader_election.cc:288] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919), cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111)
W20260812 05:29:05.582604 9880 leader_election.cc:341] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919): Not found: Tablet not found: 77220db8b3164a0694c03a866d5f96fc
W20260812 05:29:05.582823 9881 leader_election.cc:341] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111): Not found: Tablet not found: 77220db8b3164a0694c03a866d5f96fc
I20260812 05:29:05.582883 9881 leader_election.cc:302] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb207717cd0f40debf4ef346b3425be9; no voters: 35c78683293248f08e8013fcaac49603, cce176cdb26c41ce977b9428c09d9335
I20260812 05:29:05.582976 10313 raft_consensus.cc:2726] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:29:06.080658 10314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:06.080406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:60687)
0812 05:29:06.080565 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:06.080605 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260812 05:29:06.132730 10315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:06.132422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:56759)
0812 05:29:06.132618 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:06.132673 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":95,"threads_started":1}
I20260812 05:29:06.270134 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:06.269851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:56143)
0812 05:29:06.270019 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:06.270084 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260812 05:29:07.081498 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:07.081207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:58487)
0812 05:29:07.081396 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:07.081448 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":75,"threads_started":1}
I20260812 05:29:07.133579 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:07.133330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:56797)
0812 05:29:07.133482 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:07.133528 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260812 05:29:07.271148 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:07.270852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:52923)
0812 05:29:07.271042 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:07.271096 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":89,"threads_started":1}
I20260812 05:29:07.461472 10320 raft_consensus.cc:482] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:29:07.461613 10320 raft_consensus.cc:504] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35c78683293248f08e8013fcaac49603" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37919 } } peers { permanent_uuid: "cce176cdb26c41ce977b9428c09d9335" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 41111 } } peers { permanent_uuid: "cb207717cd0f40debf4ef346b3425be9" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34163 } }
I20260812 05:29:07.461889 10320 leader_election.cc:288] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919), cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111)
W20260812 05:29:07.462280 9880 leader_election.cc:341] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 35c78683293248f08e8013fcaac49603 (127.24.66.66:37919): Not found: Tablet not found: 77220db8b3164a0694c03a866d5f96fc
W20260812 05:29:07.462394 9881 leader_election.cc:341] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cce176cdb26c41ce977b9428c09d9335 (127.24.66.67:41111): Not found: Tablet not found: 77220db8b3164a0694c03a866d5f96fc
I20260812 05:29:07.462472 9881 leader_election.cc:302] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb207717cd0f40debf4ef346b3425be9; no voters: 35c78683293248f08e8013fcaac49603, cce176cdb26c41ce977b9428c09d9335
I20260812 05:29:07.462544 10320 raft_consensus.cc:2726] T 77220db8b3164a0694c03a866d5f96fc P cb207717cd0f40debf4ef346b3425be9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 05:29:08.082479 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:08.082136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:35879)
0812 05:29:08.082315 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:08.082428 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":74,"threads_started":1}
I20260812 05:29:08.134424 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:08.134099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:33695)
0812 05:29:08.134258 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:08.134306 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
I20260812 05:29:08.272143 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:08.271846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:44627)
0812 05:29:08.272013 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:08.272079 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":75,"threads_started":1}
I20260812 05:29:09.083528 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:09.083150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:50293)
0812 05:29:09.083428 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:09.083477 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":165,"threads_started":1}
I20260812 05:29:09.135264 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:09.135021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:51443)
0812 05:29:09.135160 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:09.135210 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260812 05:29:09.272989 10327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:09.272691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:44367)
0812 05:29:09.272889 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:09.272938 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260812 05:29:10.084489 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:10.084202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:55229)
0812 05:29:10.084381 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:10.084436 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":73,"threads_started":1}
I20260812 05:29:10.136215 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:10.135823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:50495)
0812 05:29:10.136016 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:10.136065 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
I20260812 05:29:10.273911 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:10.273625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:48323)
0812 05:29:10.273808 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:10.273860 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
I20260812 05:29:11.085590 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:11.085331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.65:40171)
0812 05:29:11.085496 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:11.085542 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
I20260812 05:29:11.137105 10332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:11.136845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.66:60691)
0812 05:29:11.137005 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:11.137055 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
I20260812 05:29:11.271013 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 9829
I20260812 05:29:11.275347 10333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:11.274699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:35627 (local address 127.24.66.67:36813)
0812 05:29:11.275221 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:11.275274 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.126:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":83,"threads_started":1}
I20260812 05:29:11.280447 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 10004
I20260812 05:29:11.287444 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 10144
2026-08-12T05:29:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58851 ms)
[----------] 30 tests from SecurityITest (249336 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:11 dist-test-slave-zpfg krb5kdc[10336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:11 dist-test-slave-zpfg krb5kdc[10336](info): set up 2 sockets
Aug 12 05:29:11 dist-test-slave-zpfg krb5kdc[10336](info): commencing operation
krb5kdc: starting...
W20260812 05:29:12.367650 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:12 dist-test-slave-zpfg krb5kdc[10336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:29:12.501856 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:46645
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:46645
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:12.591418 10350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:12.591651 10350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:12.591678 10350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:12.592943 10350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:12.592988 10350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:12.593004 10350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:12.593019 10350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:12.593032 10350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:12.594942 10350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:46645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:46645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:12.595204 10350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:12.595412 10350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:12.601629 10358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:12.601768 10350 server_base.cc:1034] running on GCE node
W20260812 05:29:12.601622 10356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:12.601626 10355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:12.602381 10350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:12.602726 10350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:12.603904 10350 hybrid_clock.cc:648] HybridClock initialized: now 1786512552603866 us; error 56 us; skew 500 ppm
Aug 12 05:29:12 dist-test-slave-zpfg krb5kdc[10336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:12.606716 10350 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:29:12.607223 10350 webserver.cc:460] Webserver started at http://127.24.66.126:33647/ using document root <none> and password file <none>
I20260812 05:29:12.607609 10350 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:12.607682 10350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:12.607842 10350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:12.609038 10350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "c5c882a73f564d51ba1e644a87197bff"
format_stamp: "Formatted at 2026-08-12 05:29:12 on dist-test-slave-zpfg"
server_key: "9a0bf9a141a21a64fb5e8597a37edd34"
server_key_iv: "528122537b84ae8f20ac6b5fd344727c"
server_key_version: "encryptionkey@0"
I20260812 05:29:12.609433 10350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "c5c882a73f564d51ba1e644a87197bff"
format_stamp: "Formatted at 2026-08-12 05:29:12 on dist-test-slave-zpfg"
server_key: "9a0bf9a141a21a64fb5e8597a37edd34"
server_key_iv: "528122537b84ae8f20ac6b5fd344727c"
server_key_version: "encryptionkey@0"
I20260812 05:29:12.611124 10350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 05:29:12.612087 10365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:12.612409 10350 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:12.612519 10350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "c5c882a73f564d51ba1e644a87197bff"
format_stamp: "Formatted at 2026-08-12 05:29:12 on dist-test-slave-zpfg"
server_key: "9a0bf9a141a21a64fb5e8597a37edd34"
server_key_iv: "528122537b84ae8f20ac6b5fd344727c"
server_key_version: "encryptionkey@0"
I20260812 05:29:12.612608 10350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:12.634871 10350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:12.637306 10350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:12.637501 10350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:12.641873 10350 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:46645
I20260812 05:29:12.641922 10422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:46645 every 8 connection(s)
I20260812 05:29:12.642330 10350 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:12.643345 10423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:12.645895 10423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff: Bootstrap starting.
I20260812 05:29:12.646625 10423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:12.646948 10423 log.cc:824] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:12.647759 10423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff: No bootstrap required, opened a new log
I20260812 05:29:12.648342 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 10350
I20260812 05:29:12.648427 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:12.648737 24841 external_mini_cluster.cc:1428] Setting key b021d38b6b88304ed174afbd8954f71e
I20260812 05:29:12.651093 10423 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c882a73f564d51ba1e644a87197bff" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46645 } }
I20260812 05:29:12.651292 10423 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:12.651340 10423 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c882a73f564d51ba1e644a87197bff, State: Initialized, Role: FOLLOWER
I20260812 05:29:12.651542 10423 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c882a73f564d51ba1e644a87197bff" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46645 } }
I20260812 05:29:12.651688 10423 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:12.651737 10423 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:12.651789 10423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:12.652555 10423 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c882a73f564d51ba1e644a87197bff" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46645 } }
I20260812 05:29:12.652688 10423 leader_election.cc:302] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5c882a73f564d51ba1e644a87197bff; no voters:
I20260812 05:29:12.652859 10423 leader_election.cc:288] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:12.652935 10428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:12.653141 10428 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [term 1 LEADER]: Becoming Leader. State: Replica: c5c882a73f564d51ba1e644a87197bff, State: Running, Role: LEADER
I20260812 05:29:12.653123 10423 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:12.653357 10428 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c882a73f564d51ba1e644a87197bff" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46645 } }
I20260812 05:29:12.653718 10429 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5c882a73f564d51ba1e644a87197bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c882a73f564d51ba1e644a87197bff" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46645 } } }
I20260812 05:29:12.653800 10429 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:12.653980 10430 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5c882a73f564d51ba1e644a87197bff. Latest consensus state: current_term: 1 leader_uuid: "c5c882a73f564d51ba1e644a87197bff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c882a73f564d51ba1e644a87197bff" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 46645 } } }
I20260812 05:29:12.654037 10430 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:12.654491 10434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:12.654978 10434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:12.656845 10434 catalog_manager.cc:1347] Generated new cluster ID: 008ad2e958624a3ab815d4d8a3144c43
I20260812 05:29:12.656904 10434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:29:12 dist-test-slave-zpfg krb5kdc[10336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:12.661767 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:12.649348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35294 (local address 127.24.66.126:46645)
0812 05:29:12.649559 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:12.649569 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:12.649607 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:29:12.651036 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:12.651046 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:12.651237 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:12.651404 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:12.652183 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:12.653655 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:12.654973 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:12.655670 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:12.655813 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:12.658540 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:12.658544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:12.658544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:12.658554 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:12.660186 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:12.660557 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:12.660561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:12.660561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:12.660654 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:12.660883 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:12.660884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:12.660884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:12.661127 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:12.661310 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:12.661412 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 05:29:12.661561 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
I20260812 05:29:12.681488 10434 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:12.682073 10434 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:12.686590 10434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff: Generated new TSK 0
I20260812 05:29:12.686836 10434 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:29:12.725220 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:46645
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:12.814579 10450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:12.814800 10450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:12.814824 10450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:12.816082 10450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:12.816143 10450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:12.816171 10450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:29:12.817780 10450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:46645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:12.818059 10450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:12.818324 10450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:29:12.823752 10450 server_base.cc:1034] running on GCE node
W20260812 05:29:12.823720 10455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:12.823837 10456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:12.823877 10458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:12.824338 10450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:12.824628 10450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:12.825836 10450 hybrid_clock.cc:648] HybridClock initialized: now 1786512552825813 us; error 51 us; skew 500 ppm
Aug 12 05:29:12 dist-test-slave-zpfg krb5kdc[10336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:12.829002 10450 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:29:12.829783 10450 webserver.cc:460] Webserver started at http://127.24.66.65:45303/ using document root <none> and password file <none>
I20260812 05:29:12.830168 10450 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:12.830283 10450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:12.830492 10450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:12.831770 10450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "653b015ed42a418c958e7a421f227a92"
format_stamp: "Formatted at 2026-08-12 05:29:12 on dist-test-slave-zpfg"
server_key: "ed8928ed65f6a7a1b9689f1fe58be596"
server_key_iv: "f4896f12b411250326147cf198596a42"
server_key_version: "encryptionkey@0"
I20260812 05:29:12.832193 10450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "653b015ed42a418c958e7a421f227a92"
format_stamp: "Formatted at 2026-08-12 05:29:12 on dist-test-slave-zpfg"
server_key: "ed8928ed65f6a7a1b9689f1fe58be596"
server_key_iv: "f4896f12b411250326147cf198596a42"
server_key_version: "encryptionkey@0"
I20260812 05:29:12.833833 10450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:29:12.834682 10465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:12.834854 10450 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:12.834960 10450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "653b015ed42a418c958e7a421f227a92"
format_stamp: "Formatted at 2026-08-12 05:29:12 on dist-test-slave-zpfg"
server_key: "ed8928ed65f6a7a1b9689f1fe58be596"
server_key_iv: "f4896f12b411250326147cf198596a42"
server_key_version: "encryptionkey@0"
I20260812 05:29:12.835048 10450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:12.856983 10450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:12.859917 10450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:12.860126 10450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:12.860424 10450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:12.860822 10450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:12.860888 10450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:12.860929 10450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:12.860988 10450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:12.868281 10450 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:40559
I20260812 05:29:12.868330 10583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:40559 every 8 connection(s)
I20260812 05:29:12.868690 10450 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:29:12.871781 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 10450
I20260812 05:29:12.871917 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:29:12.872315 24841 external_mini_cluster.cc:1428] Setting key c7a302c74fdc8d8b9342b535cfa1cfbc
Aug 12 05:29:12 dist-test-slave-zpfg krb5kdc[10336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:12.880077 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:12.869514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:46645 (local address 127.24.66.65:55267)
0812 05:29:12.869696 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:12.869741 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:12.870247 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:12.870510 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:12.870514 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:12.871115 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:12.871322 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:12.871327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:12.872087 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:12.872089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:12.873326 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:12.873329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:12.873718 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:12.873719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:12.873866 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:12.874536 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:12.874547 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:12.876487 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:12.878453 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:12.878460 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:12.878462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:12.878701 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:12.879006 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:12.879007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:12.879007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:12.879137 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:12.879509 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:12.879515 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:12.879643 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 05:29:12.879764 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 05:29:12.879903 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260812 05:29:12.880129 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:12.869550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:55267 (local address 127.24.66.126:46645)
0812 05:29:12.869617 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:12.869619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:12.869829 (+ 210us) server_negotiation.cc:407] Connection header received
0812 05:29:12.870330 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:12.870331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:12.870365 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:12.870427 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:12.871407 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:12.871992 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:12.873422 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:12.873620 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:12.873678 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:12.876591 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:12.876592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:12.876592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:12.876601 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:12.878331 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:12.878806 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:12.878808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:12.878808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:12.878892 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:12.879241 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:12.879243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:12.879244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:12.879420 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:12.879517 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:12.879844 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 05:29:12.879972 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:29:12.881003 10584 heartbeater.cc:347] Connected to a master server at 127.24.66.126:46645
I20260812 05:29:12.881136 10584 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:12.881362 10584 heartbeater.cc:511] Master 127.24.66.126:46645 requested a full tablet report, sending...
I20260812 05:29:12.882046 10383 ts_manager.cc:194] Registered new tserver with Master: 653b015ed42a418c958e7a421f227a92 (127.24.66.65:40559)
I20260812 05:29:12.883142 10383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:55267
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:29:12.906527 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:46645
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:12.993594 10589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:12.993861 10589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:12.993901 10589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:12.995378 10589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:12.995463 10589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:12.995510 10589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:29:12.997058 10589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:46645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:12.997363 10589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:12.997618 10589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:13.003284 10598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:13.003306 10589 server_base.cc:1034] running on GCE node
W20260812 05:29:13.003273 10596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:13.003273 10595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:13.003809 10589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:13.004086 10589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:13.005292 10589 hybrid_clock.cc:648] HybridClock initialized: now 1786512553005251 us; error 58 us; skew 500 ppm
Aug 12 05:29:13 dist-test-slave-zpfg krb5kdc[10336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512553, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:13.008136 10589 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:29:13.008653 10589 webserver.cc:460] Webserver started at http://127.24.66.66:44265/ using document root <none> and password file <none>
I20260812 05:29:13.008873 10589 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:13.008917 10589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:13.009014 10589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:13.010069 10589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "1d37be3510998e6deadb13ef2a6938b1"
server_key_iv: "5e8dc2860e337a85acae1f42aaf0f985"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.010458 10589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "1d37be3510998e6deadb13ef2a6938b1"
server_key_iv: "5e8dc2860e337a85acae1f42aaf0f985"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.011864 10589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:13.012678 10605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:13.012995 10589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:13.013116 10589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "1d37be3510998e6deadb13ef2a6938b1"
server_key_iv: "5e8dc2860e337a85acae1f42aaf0f985"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.013200 10589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:13.026774 10589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:13.029191 10589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:13.029347 10589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:13.029635 10589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:13.029973 10589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:13.030030 10589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:13.030071 10589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:13.030099 10589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:13.036115 10589 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:37247
I20260812 05:29:13.036194 10726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:37247 every 8 connection(s)
I20260812 05:29:13.036540 10589 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:29:13 dist-test-slave-zpfg krb5kdc[10336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512553, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:13.043627 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 10589
I20260812 05:29:13.043753 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:29:13.043975 24841 external_mini_cluster.cc:1428] Setting key 371d941f3ab3a447c0f139c50043129b
I20260812 05:29:13.047122 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.037311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:41061 (local address 127.24.66.126:46645)
0812 05:29:13.037388 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:13.037390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:13.037605 (+ 215us) server_negotiation.cc:407] Connection header received
0812 05:29:13.038213 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:13.038215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:13.038227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:13.038285 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:13.039316 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.039835 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.040847 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.041058 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.041143 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.043537 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:13.043538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:13.043539 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:13.043547 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:13.045420 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:13.045858 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:13.045860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:13.045860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:13.045923 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:13.046323 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:13.046324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:13.046324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:13.046484 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:13.046563 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:13.046962 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 05:29:13.047047 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 05:29:13.047380 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.037283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:46645 (local address 127.24.66.66:41061)
0812 05:29:13.037482 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:13.037519 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:13.038103 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:13.038389 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:13.038393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:13.038964 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:13.039210 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.039216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.039931 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.039932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.040751 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.040754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.041154 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.041155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.041306 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.041843 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:13.041851 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:13.043418 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:13.045511 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:13.045518 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:13.045521 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:13.045771 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:13.046007 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:13.046010 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:13.046010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:13.046122 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:13.046554 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:13.046561 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:13.046755 (+ 194us) client_negotiation.cc:769] Sending connection context
0812 05:29:13.046889 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 05:29:13.047084 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":96,"threads_started":1}
I20260812 05:29:13.048018 10727 heartbeater.cc:347] Connected to a master server at 127.24.66.126:46645
I20260812 05:29:13.048120 10727 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:13.048346 10727 heartbeater.cc:511] Master 127.24.66.126:46645 requested a full tablet report, sending...
I20260812 05:29:13.048800 10383 ts_manager.cc:194] Registered new tserver with Master: 0a7a72b0fc7a4ca0bd06e8c39886fa52 (127.24.66.66:37247)
I20260812 05:29:13.049260 10383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:41061
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:29:13.074304 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:46645
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:13.161551 10732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:13.161785 10732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:13.161810 10732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:13.163079 10732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:13.163133 10732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:13.163162 10732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:29:13.164685 10732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:46645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:13.164939 10732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:13.165203 10732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:13.170830 10737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:13.170854 10740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:13.170997 10732 server_base.cc:1034] running on GCE node
W20260812 05:29:13.170850 10738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:13.171358 10732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:13.171590 10732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:13.172744 10732 hybrid_clock.cc:648] HybridClock initialized: now 1786512553172734 us; error 34 us; skew 500 ppm
Aug 12 05:29:13 dist-test-slave-zpfg krb5kdc[10336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512553, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:13.175791 10732 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:29:13.176290 10732 webserver.cc:460] Webserver started at http://127.24.66.67:38967/ using document root <none> and password file <none>
I20260812 05:29:13.176517 10732 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:13.176561 10732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:13.176730 10732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:13.177824 10732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "87e6ca4cdf33420295324e87b862efa6"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "7af15d5547787c96ec32e8f6b03623aa"
server_key_iv: "e99e7aa44aaec73c38604837fe7ff00c"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.178277 10732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "87e6ca4cdf33420295324e87b862efa6"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "7af15d5547787c96ec32e8f6b03623aa"
server_key_iv: "e99e7aa44aaec73c38604837fe7ff00c"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.179678 10732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:13.180539 10747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:13.180727 10732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:13.180832 10732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "87e6ca4cdf33420295324e87b862efa6"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "7af15d5547787c96ec32e8f6b03623aa"
server_key_iv: "e99e7aa44aaec73c38604837fe7ff00c"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.180907 10732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:13.196470 10732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:13.199101 10732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:13.199313 10732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:13.199533 10732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:13.199895 10732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:13.199957 10732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:13.199995 10732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:13.200043 10732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:13.206286 10732 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:35591
I20260812 05:29:13.206461 10864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:35591 every 8 connection(s)
I20260812 05:29:13.206758 10732 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:29:13.210843 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 10732
I20260812 05:29:13.210942 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:29:13.211117 24841 external_mini_cluster.cc:1428] Setting key 50db777f6d5256bcc618c2dc9a1c0980
Aug 12 05:29:13 dist-test-slave-zpfg krb5kdc[10336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512553, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:13.217166 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.207673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:55631 (local address 127.24.66.126:46645)
0812 05:29:13.207759 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:13.207761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:13.207932 (+ 171us) server_negotiation.cc:407] Connection header received
0812 05:29:13.208496 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:13.208497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:13.208507 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:13.208562 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:13.209541 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.210057 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.211498 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.211784 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.211860 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.214160 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:13.214162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:13.214163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:13.214171 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:13.215595 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:13.216127 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:13.216128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:13.216128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:13.216191 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:13.216482 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:13.216483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:13.216483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:13.216613 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:13.216689 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:13.216983 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 05:29:13.217069 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 05:29:13.217391 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.207617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:46645 (local address 127.24.66.67:55631)
0812 05:29:13.207799 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:13.207844 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:13.208397 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:13.208623 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:13.208626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:13.209240 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:13.209446 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.209451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.210156 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.210157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.211405 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.211408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.211922 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.211923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.212041 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.212550 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:13.212558 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:13.214056 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:13.215705 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:13.215713 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:13.215715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:13.216031 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:13.216270 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:13.216271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:13.216271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:13.216409 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:13.216689 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:13.216697 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:13.216842 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 05:29:13.216949 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 05:29:13.217133 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260812 05:29:13.217940 10865 heartbeater.cc:347] Connected to a master server at 127.24.66.126:46645
I20260812 05:29:13.218055 10865 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:13.218252 10865 heartbeater.cc:511] Master 127.24.66.126:46645 requested a full tablet report, sending...
I20260812 05:29:13.218668 10383 ts_manager.cc:194] Registered new tserver with Master: 87e6ca4cdf33420295324e87b862efa6 (127.24.66.67:35591)
I20260812 05:29:13.219038 10383 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:55631
I20260812 05:29:13.223135 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:29:13.231383 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.224584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35300 (local address 127.24.66.126:46645)
0812 05:29:13.224648 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:13.224650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:13.224775 (+ 125us) server_negotiation.cc:407] Connection header received
0812 05:29:13.224826 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:13.224826 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:13.224835 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:13.224881 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:13.225530 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.226007 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.226958 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.227143 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.227201 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.228605 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:13.228606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:13.228606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:13.228612 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:13.229972 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:13.230558 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:13.230559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:13.230560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:13.230616 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:13.230794 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:13.230795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:13.230795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:13.230946 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:13.231028 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:13.231203 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 05:29:13.231308 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 05:29:13.233359 10383 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:29:13.235134 10383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:29:13.242767 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.238425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.24.66.66:37247)
0812 05:29:13.238625 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:13.238630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:13.238728 (+ 98us) server_negotiation.cc:407] Connection header received
0812 05:29:13.238779 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:13.238782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:13.238802 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:13.238889 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:13.239189 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.240283 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.241508 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.242227 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.242320 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.242409 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:13.242617 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 05:29:13.242665 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260812 05:29:13.242877 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.238062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:40559 (local address 127.0.0.1:38342)
0812 05:29:13.238271 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:13.238316 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:13.238414 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:13.238899 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:13.238903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:13.238905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:13.238981 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.238983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.240415 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.240416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.241661 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.241663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.242503 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.242504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.242584 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.242588 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:29:13.242638 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 05:29:13.242684 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":4,"thread_start_us":80,"threads_started":1}
I20260812 05:29:13.243041 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.238427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.24.66.67:35591)
0812 05:29:13.238849 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:13.238854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:13.238865 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:29:13.238906 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:13.238908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:13.238923 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:13.238999 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:13.239211 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.239978 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.241734 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.242511 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.242579 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.242635 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:13.242912 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 05:29:13.242961 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":78,"threads_started":1}
I20260812 05:29:13.242933 10880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.238201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38342 (local address 127.24.66.65:40559)
0812 05:29:13.238641 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:13.238646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:13.238656 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:29:13.238752 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:13.238754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:13.238771 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:13.239062 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:13.239159 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.240299 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.241768 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.242407 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.242489 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.242555 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:13.242803 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 05:29:13.242852 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":92,"threads_started":1}
I20260812 05:29:13.242758 10879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.238085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:37247 (local address 127.0.0.1:38314)
0812 05:29:13.238562 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:13.238571 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:13.238611 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:13.238983 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:13.238984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:13.238987 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:13.239090 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.239092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.240427 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.240428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.241414 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.241416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.242371 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.242372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.242473 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.242492 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 05:29:13.242572 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 05:29:13.242634 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":45,"thread_start_us":109,"threads_started":1}
I20260812 05:29:13.243495 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.238085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:35591 (local address 127.0.0.1:51194)
0812 05:29:13.238417 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:13.238429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:13.238478 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:13.239085 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:13.239086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:13.239088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:13.239129 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.239130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.240092 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.240093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.241627 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.241630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.242666 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.242667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.242722 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.242724 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:29:13.242785 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 05:29:13.242820 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":67,"thread_start_us":129,"threads_started":1}
I20260812 05:29:13.243849 10514 tablet_service.cc:1467] Processing CreateTablet for tablet 64677b26b1404e578c1aa90ff74e0217 (DEFAULT_TABLE table=test-table [id=a006fa5f5b9548369851cf3c71a09254]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:13.244241 10514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64677b26b1404e578c1aa90ff74e0217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:13.244191 10796 tablet_service.cc:1467] Processing CreateTablet for tablet 64677b26b1404e578c1aa90ff74e0217 (DEFAULT_TABLE table=test-table [id=a006fa5f5b9548369851cf3c71a09254]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:13.244360 10655 tablet_service.cc:1467] Processing CreateTablet for tablet 64677b26b1404e578c1aa90ff74e0217 (DEFAULT_TABLE table=test-table [id=a006fa5f5b9548369851cf3c71a09254]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:13.244520 10796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64677b26b1404e578c1aa90ff74e0217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:13.244660 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64677b26b1404e578c1aa90ff74e0217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:13.247710 10883 tablet_bootstrap.cc:492] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: Bootstrap starting.
I20260812 05:29:13.248224 10884 tablet_bootstrap.cc:492] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: Bootstrap starting.
I20260812 05:29:13.248450 10883 tablet_bootstrap.cc:654] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:13.248963 10884 tablet_bootstrap.cc:654] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:13.249161 10883 log.cc:824] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:13.249573 10884 log.cc:824] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:13.249948 10883 tablet_bootstrap.cc:492] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: No bootstrap required, opened a new log
I20260812 05:29:13.249946 10885 tablet_bootstrap.cc:492] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92: Bootstrap starting.
I20260812 05:29:13.250041 10883 ts_tablet_manager.cc:1397] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:29:13.250653 10885 tablet_bootstrap.cc:654] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:13.250746 10884 tablet_bootstrap.cc:492] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: No bootstrap required, opened a new log
I20260812 05:29:13.250857 10884 ts_tablet_manager.cc:1397] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:29:13.251294 10885 log.cc:824] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:13.251996 10883 raft_consensus.cc:348] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653b015ed42a418c958e7a421f227a92" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40559 } } peers { permanent_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37247 } } peers { permanent_uuid: "87e6ca4cdf33420295324e87b862efa6" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35591 } }
I20260812 05:29:13.252205 10885 tablet_bootstrap.cc:492] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92: No bootstrap required, opened a new log
I20260812 05:29:13.252224 10883 raft_consensus.cc:374] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:13.252266 10885 ts_tablet_manager.cc:1397] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:29:13.252277 10883 raft_consensus.cc:729] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a7a72b0fc7a4ca0bd06e8c39886fa52, State: Initialized, Role: FOLLOWER
I20260812 05:29:13.252413 10883 consensus_queue.cc:260] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653b015ed42a418c958e7a421f227a92" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40559 } } peers { permanent_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37247 } } peers { permanent_uuid: "87e6ca4cdf33420295324e87b862efa6" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35591 } }
I20260812 05:29:13.252696 10883 ts_tablet_manager.cc:1428] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:29:13.252731 10727 heartbeater.cc:503] Master 127.24.66.126:46645 was elected leader, sending a full tablet report...
I20260812 05:29:13.252879 10884 raft_consensus.cc:348] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653b015ed42a418c958e7a421f227a92" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40559 } } peers { permanent_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37247 } } peers { permanent_uuid: "87e6ca4cdf33420295324e87b862efa6" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35591 } }
I20260812 05:29:13.253036 10884 raft_consensus.cc:374] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:13.253103 10884 raft_consensus.cc:729] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e6ca4cdf33420295324e87b862efa6, State: Initialized, Role: FOLLOWER
I20260812 05:29:13.253257 10884 consensus_queue.cc:260] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653b015ed42a418c958e7a421f227a92" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40559 } } peers { permanent_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37247 } } peers { permanent_uuid: "87e6ca4cdf33420295324e87b862efa6" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35591 } }
I20260812 05:29:13.253552 10865 heartbeater.cc:503] Master 127.24.66.126:46645 was elected leader, sending a full tablet report...
I20260812 05:29:13.253552 10884 ts_tablet_manager.cc:1428] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:29:13.254397 10885 raft_consensus.cc:348] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653b015ed42a418c958e7a421f227a92" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40559 } } peers { permanent_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37247 } } peers { permanent_uuid: "87e6ca4cdf33420295324e87b862efa6" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35591 } }
I20260812 05:29:13.254567 10885 raft_consensus.cc:374] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:13.254599 10885 raft_consensus.cc:729] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 653b015ed42a418c958e7a421f227a92, State: Initialized, Role: FOLLOWER
I20260812 05:29:13.254721 10885 consensus_queue.cc:260] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653b015ed42a418c958e7a421f227a92" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40559 } } peers { permanent_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37247 } } peers { permanent_uuid: "87e6ca4cdf33420295324e87b862efa6" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35591 } }
I20260812 05:29:13.255004 10885 ts_tablet_manager.cc:1428] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:29:13.255080 10584 heartbeater.cc:503] Master 127.24.66.126:46645 was elected leader, sending a full tablet report...
W20260812 05:29:13.287117 10728 tablet.cc:2369] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:29:13.369552 10585 tablet.cc:2369] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:29:13.457401 10866 tablet.cc:2369] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:29:13.555985 10891 raft_consensus.cc:482] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:29:13.556146 10891 raft_consensus.cc:504] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653b015ed42a418c958e7a421f227a92" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40559 } } peers { permanent_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37247 } } peers { permanent_uuid: "87e6ca4cdf33420295324e87b862efa6" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35591 } }
I20260812 05:29:13.556708 10891 leader_election.cc:288] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a7a72b0fc7a4ca0bd06e8c39886fa52 (127.24.66.66:37247), 87e6ca4cdf33420295324e87b862efa6 (127.24.66.67:35591)
I20260812 05:29:13.560002 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.557019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:54617 (local address 127.24.66.66:37247)
0812 05:29:13.557072 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:13.557074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:13.557184 (+ 110us) server_negotiation.cc:407] Connection header received
0812 05:29:13.557275 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:13.557276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:13.557287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:13.557370 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:13.557548 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.558042 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.559254 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.559620 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.559677 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.559684 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:13.559877 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 05:29:13.559926 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:29:13.560088 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.557054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:53839 (local address 127.24.66.67:35591)
0812 05:29:13.557107 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:13.557109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:13.557197 (+ 88us) server_negotiation.cc:407] Connection header received
0812 05:29:13.557262 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:13.557263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:13.557274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:13.557336 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:13.557595 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.558141 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.559230 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.559688 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.559769 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.559778 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:13.559954 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 05:29:13.560002 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:29:13.560042 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.556947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:37247 (local address 127.24.66.65:54617)
0812 05:29:13.557088 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:13.557101 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:13.557201 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:13.557397 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:13.557398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:13.557401 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:13.557476 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.557478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.558130 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.558131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.559133 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.559136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.559717 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.559719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.559800 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.559804 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:29:13.559868 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 05:29:13.559906 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":5,"thread_start_us":66,"threads_started":1}
I20260812 05:29:13.560072 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.556951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:35591 (local address 127.24.66.65:53839)
0812 05:29:13.557101 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:13.557110 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:13.557199 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:13.557363 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:13.557365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:13.557368 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:13.557493 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.557495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.558295 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.558296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.559130 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.559132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.559785 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.559786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.559870 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.559874 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:29:13.559932 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 05:29:13.559970 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260812 05:29:13.560472 10677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64677b26b1404e578c1aa90ff74e0217" candidate_uuid: "653b015ed42a418c958e7a421f227a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" is_pre_election: true
I20260812 05:29:13.560678 10677 raft_consensus.cc:2454] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 653b015ed42a418c958e7a421f227a92 in term 0.
I20260812 05:29:13.560696 10817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64677b26b1404e578c1aa90ff74e0217" candidate_uuid: "653b015ed42a418c958e7a421f227a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e6ca4cdf33420295324e87b862efa6" is_pre_election: true
I20260812 05:29:13.560868 10817 raft_consensus.cc:2454] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 653b015ed42a418c958e7a421f227a92 in term 0.
I20260812 05:29:13.560957 10468 leader_election.cc:302] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a7a72b0fc7a4ca0bd06e8c39886fa52, 653b015ed42a418c958e7a421f227a92; no voters:
I20260812 05:29:13.561195 10891 raft_consensus.cc:2781] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:29:13.561267 10891 raft_consensus.cc:482] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:29:13.561307 10891 raft_consensus.cc:3037] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:13.562108 10891 raft_consensus.cc:504] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653b015ed42a418c958e7a421f227a92" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40559 } } peers { permanent_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37247 } } peers { permanent_uuid: "87e6ca4cdf33420295324e87b862efa6" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35591 } }
I20260812 05:29:13.562291 10891 leader_election.cc:288] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [CANDIDATE]: Term 1 election: Requested vote from peers 0a7a72b0fc7a4ca0bd06e8c39886fa52 (127.24.66.66:37247), 87e6ca4cdf33420295324e87b862efa6 (127.24.66.67:35591)
I20260812 05:29:13.562517 10817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64677b26b1404e578c1aa90ff74e0217" candidate_uuid: "653b015ed42a418c958e7a421f227a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e6ca4cdf33420295324e87b862efa6"
I20260812 05:29:13.562546 10677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64677b26b1404e578c1aa90ff74e0217" candidate_uuid: "653b015ed42a418c958e7a421f227a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52"
I20260812 05:29:13.562616 10817 raft_consensus.cc:3037] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:13.562650 10677 raft_consensus.cc:3037] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:13.563452 10677 raft_consensus.cc:2454] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 653b015ed42a418c958e7a421f227a92 in term 1.
I20260812 05:29:13.563462 10817 raft_consensus.cc:2454] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 653b015ed42a418c958e7a421f227a92 in term 1.
I20260812 05:29:13.563657 10466 leader_election.cc:302] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 653b015ed42a418c958e7a421f227a92, 87e6ca4cdf33420295324e87b862efa6; no voters:
I20260812 05:29:13.563783 10891 raft_consensus.cc:2781] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:13.563951 10891 raft_consensus.cc:686] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [term 1 LEADER]: Becoming Leader. State: Replica: 653b015ed42a418c958e7a421f227a92, State: Running, Role: LEADER
I20260812 05:29:13.564038 10891 consensus_queue.cc:237] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653b015ed42a418c958e7a421f227a92" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40559 } } peers { permanent_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37247 } } peers { permanent_uuid: "87e6ca4cdf33420295324e87b862efa6" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35591 } }
I20260812 05:29:13.565058 10382 catalog_manager.cc:5495] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 reported cstate change: term changed from 0 to 1, leader changed from <none> to 653b015ed42a418c958e7a421f227a92 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "653b015ed42a418c958e7a421f227a92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "653b015ed42a418c958e7a421f227a92" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e6ca4cdf33420295324e87b862efa6" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35591 } health_report { overall_health: UNKNOWN } } }
I20260812 05:29:13.599426 10880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.596723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.24.66.65:40559)
0812 05:29:13.596819 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:13.596820 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:13.596829 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:29:13.596868 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:13.596869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:13.596889 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:13.596954 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:13.597128 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.597598 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.598632 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.598832 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.598889 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.599101 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:13.599214 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:13.599295 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:13.599340 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 05:29:13.599361 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 05:29:13.601668 10817 raft_consensus.cc:1261] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6 [term 1 FOLLOWER]: Refusing update from remote peer 653b015ed42a418c958e7a421f227a92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:13.601668 10677 raft_consensus.cc:1261] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52 [term 1 FOLLOWER]: Refusing update from remote peer 653b015ed42a418c958e7a421f227a92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:13.602144 10891 consensus_queue.cc:1035] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87e6ca4cdf33420295324e87b862efa6" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:13.602321 10894 consensus_queue.cc:1035] T 64677b26b1404e578c1aa90ff74e0217 P 653b015ed42a418c958e7a421f227a92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a7a72b0fc7a4ca0bd06e8c39886fa52" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:13.605894 10899 mvcc.cc:205] Tried to move back new op lower bound from 7317555419548651520 to 7317555419399069696. Current Snapshot: MvccSnapshot[applied={T|T < 7317555419548651520}]
I20260812 05:29:13.607743 10382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:13.607864 10382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:13.608753 10382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5c882a73f564d51ba1e644a87197bff: Sending DeleteTablet for 3 replicas of tablet 64677b26b1404e578c1aa90ff74e0217
I20260812 05:29:13.609102 10514 tablet_service.cc:1514] Processing DeleteTablet for tablet 64677b26b1404e578c1aa90ff74e0217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:13 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:38342
I20260812 05:29:13.609133 10796 tablet_service.cc:1514] Processing DeleteTablet for tablet 64677b26b1404e578c1aa90ff74e0217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:13 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:51194
I20260812 05:29:13.609165 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet 64677b26b1404e578c1aa90ff74e0217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:13 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:38314
I20260812 05:29:13.609367 10907 tablet_replica.cc:330] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: stopping tablet replica
I20260812 05:29:13.609527 10907 raft_consensus.cc:2229] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:29:13.609618 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 10450
I20260812 05:29:13.609694 10907 raft_consensus.cc:2258] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:13.610675 10909 tablet_replica.cc:330] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: stopping tablet replica
I20260812 05:29:13.610770 10909 raft_consensus.cc:2229] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:29:13.610862 10909 raft_consensus.cc:2258] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:13.611992 10909 ts_tablet_manager.cc:1905] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:13.612888 10907 ts_tablet_manager.cc:1905] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:13.614310 10909 ts_tablet_manager.cc:1918] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:13.614461 10909 log.cc:1196] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/64677b26b1404e578c1aa90ff74e0217
I20260812 05:29:13.614486 10907 ts_tablet_manager.cc:1918] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:13.614609 10907 log.cc:1196] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/64677b26b1404e578c1aa90ff74e0217
I20260812 05:29:13.614821 10909 ts_tablet_manager.cc:1939] T 64677b26b1404e578c1aa90ff74e0217 P 0a7a72b0fc7a4ca0bd06e8c39886fa52: Deleting consensus metadata
I20260812 05:29:13.615363 10369 catalog_manager.cc:4841] TS 0a7a72b0fc7a4ca0bd06e8c39886fa52 (127.24.66.66:37247): tablet 64677b26b1404e578c1aa90ff74e0217 (table test-table [id=a006fa5f5b9548369851cf3c71a09254]) successfully deleted
I20260812 05:29:13.615568 10907 ts_tablet_manager.cc:1939] T 64677b26b1404e578c1aa90ff74e0217 P 87e6ca4cdf33420295324e87b862efa6: Deleting consensus metadata
I20260812 05:29:13.616225 10366 catalog_manager.cc:4841] TS 87e6ca4cdf33420295324e87b862efa6 (127.24.66.67:35591): tablet 64677b26b1404e578c1aa90ff74e0217 (table test-table [id=a006fa5f5b9548369851cf3c71a09254]) successfully deleted
W20260812 05:29:13.618793 10370 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:40559 (error 108)
I20260812 05:29:13.619537 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 10589
I20260812 05:29:13.619563 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.619376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:40559 (local address 127.0.0.1:38360)
0812 05:29:13.619470 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:13.619523 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:40559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 05:29:13.621881 10370 catalog_manager.cc:4584] TS 653b015ed42a418c958e7a421f227a92 (127.24.66.65:40559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64677b26b1404e578c1aa90ff74e0217: Network error: Client connection negotiation failed: client connection to 127.24.66.65:40559: connect: Connection refused (error 111)
I20260812 05:29:13.626421 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 10732
I20260812 05:29:13.633234 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 10350
2026-08-12T05:29:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2345 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T05:29:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:13Z Disabled control of system clock
I20260812 05:29:13.665899 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40451
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:34581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:40451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:29:13.753813 10914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:13.754081 10914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:13.754105 10914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:13.755497 10914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:13.755550 10914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:13.755571 10914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:13.755586 10914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:13.755600 10914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:13.757102 10914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:40451
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:13.757387 10914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:13.757759 10914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:13.763211 10922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:13.763229 10919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:13.763491 10914 server_base.cc:1034] running on GCE node
W20260812 05:29:13.763263 10920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:13.763863 10914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:13.764184 10914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:13.765391 10914 hybrid_clock.cc:648] HybridClock initialized: now 1786512553765382 us; error 54 us; skew 500 ppm
I20260812 05:29:13.780634 10914 webserver.cc:460] Webserver started at http://127.24.66.126:36145/ using document root <none> and password file <none>
I20260812 05:29:13.780988 10914 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:13.781081 10914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:13.781347 10914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:13.782644 10914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "9a172761b37a4d5cbcc0bdd5e9f87669"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "4e55629a835a1634151191df53319803"
server_key_iv: "5570870b71cf9d61f756aa661b7611d6"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.783030 10914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "9a172761b37a4d5cbcc0bdd5e9f87669"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "4e55629a835a1634151191df53319803"
server_key_iv: "5570870b71cf9d61f756aa661b7611d6"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.784556 10914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:29:13.785676 10928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:13.785928 10914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:13.786051 10914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "9a172761b37a4d5cbcc0bdd5e9f87669"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "4e55629a835a1634151191df53319803"
server_key_iv: "5570870b71cf9d61f756aa661b7611d6"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.786139 10914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:13.795957 10914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:13.796337 10914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:13.796500 10914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:13.801201 10914 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:40451
I20260812 05:29:13.801335 10982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:40451 every 8 connection(s)
I20260812 05:29:13.801683 10914 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:13.802955 10983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:13.805568 10983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669: Bootstrap starting.
I20260812 05:29:13.806186 10983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:13.806527 10983 log.cc:824] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:13.807453 10983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669: No bootstrap required, opened a new log
I20260812 05:29:13.809132 10983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a172761b37a4d5cbcc0bdd5e9f87669" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40451 } }
I20260812 05:29:13.809283 10983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:13.809306 10983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a172761b37a4d5cbcc0bdd5e9f87669, State: Initialized, Role: FOLLOWER
I20260812 05:29:13.809463 10983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a172761b37a4d5cbcc0bdd5e9f87669" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40451 } }
I20260812 05:29:13.809549 10983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:13.809574 10983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:13.809603 10983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:13.810120 10983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a172761b37a4d5cbcc0bdd5e9f87669" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40451 } }
I20260812 05:29:13.810212 10983 leader_election.cc:302] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a172761b37a4d5cbcc0bdd5e9f87669; no voters:
I20260812 05:29:13.810400 10983 leader_election.cc:288] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:13.810470 10988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:13.810703 10983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:13.810798 10988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [term 1 LEADER]: Becoming Leader. State: Replica: 9a172761b37a4d5cbcc0bdd5e9f87669, State: Running, Role: LEADER
I20260812 05:29:13.810930 10988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a172761b37a4d5cbcc0bdd5e9f87669" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40451 } }
I20260812 05:29:13.811502 10991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a172761b37a4d5cbcc0bdd5e9f87669. Latest consensus state: current_term: 1 leader_uuid: "9a172761b37a4d5cbcc0bdd5e9f87669" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a172761b37a4d5cbcc0bdd5e9f87669" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40451 } } }
I20260812 05:29:13.811555 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 10914
I20260812 05:29:13.811611 10991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:13.811678 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:13.811887 10989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a172761b37a4d5cbcc0bdd5e9f87669" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a172761b37a4d5cbcc0bdd5e9f87669" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40451 } } }
I20260812 05:29:13.811940 24841 external_mini_cluster.cc:1428] Setting key 647f48b0a9703c1e3f3bbbf5791bb229
I20260812 05:29:13.812040 10989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669 [sys.catalog]: This master's current role is: LEADER
W20260812 05:29:13.813097 11005 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:13.813280 11005 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:13.813378 10998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:13.813539 10998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:13.815459 10998 catalog_manager.cc:1347] Generated new cluster ID: 62ed929419784543ae0d9a9bd858a45e
I20260812 05:29:13.815524 10998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:13.818368 11007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.812663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58250 (local address 127.24.66.126:40451)
0812 05:29:13.812878 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:13.812887 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:13.812923 (+ 36us) server_negotiation.cc:407] Connection header received
0812 05:29:13.813027 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:13.813033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:13.813092 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:13.813315 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:13.813567 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.814789 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.816011 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.816673 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.816823 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.817582 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:13.817586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:13.817586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:13.817814 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:13.817859 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:13.817878 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:13.817954 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:13.818025 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 05:29:13.818105 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260812 05:29:13.823657 10998 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:13.824259 10998 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:13.838374 10998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669: Generated new TSK 0
I20260812 05:29:13.838656 10998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:13.848690 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40451
--builtin_ntp_servers=127.24.66.84:34581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:29:13.935158 11010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:13.935415 11010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:13.935456 11010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:13.936698 11010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:13.936785 11010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:13.936836 11010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:29:13.938362 11010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:13.938668 11010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:13.939009 11010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:13.945557 11016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:13.945557 11018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:13.945724 11015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:13.945799 11010 server_base.cc:1034] running on GCE node
I20260812 05:29:13.946082 11010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:13.946391 11010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:13.947559 11010 hybrid_clock.cc:648] HybridClock initialized: now 1786512553947543 us; error 34 us; skew 500 ppm
I20260812 05:29:13.948938 11010 webserver.cc:460] Webserver started at http://127.24.66.65:36763/ using document root <none> and password file <none>
I20260812 05:29:13.949237 11010 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:13.949313 11010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:13.949467 11010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:13.950681 11010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "392c49f0e0da4c64aa73799ed1cf5a24"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "0dab8551e45285fa2b3466845aea318c"
server_key_iv: "38dc31213d0a8388f96dd63c27f67040"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.951051 11010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "392c49f0e0da4c64aa73799ed1cf5a24"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "0dab8551e45285fa2b3466845aea318c"
server_key_iv: "38dc31213d0a8388f96dd63c27f67040"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.952500 11010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:13.953385 11025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:13.953635 11010 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:13.953763 11010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "392c49f0e0da4c64aa73799ed1cf5a24"
format_stamp: "Formatted at 2026-08-12 05:29:13 on dist-test-slave-zpfg"
server_key: "0dab8551e45285fa2b3466845aea318c"
server_key_iv: "38dc31213d0a8388f96dd63c27f67040"
server_key_version: "encryptionkey@0"
I20260812 05:29:13.953845 11010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:13.982007 11010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:13.982486 11010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:13.982662 11010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:13.982975 11010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:13.983351 11010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:13.983409 11010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:13.983451 11010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:13.983479 11010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:13.989828 11010 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:45649
I20260812 05:29:13.989884 11145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:45649 every 8 connection(s)
I20260812 05:29:13.990265 11010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:29:13.995252 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 11010
I20260812 05:29:13.995344 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:29:13.995563 24841 external_mini_cluster.cc:1428] Setting key 2781af7bce78afd0011e4cae70c01ba6
I20260812 05:29:13.996577 11007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.991142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:40623 (local address 127.24.66.126:40451)
0812 05:29:13.991194 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:13.991197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:13.991508 (+ 311us) server_negotiation.cc:407] Connection header received
0812 05:29:13.992111 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:13.992113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:13.992129 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:13.992197 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:13.993018 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.993610 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.994803 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.995057 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.995147 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.996245 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:13.996247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:13.996247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:13.996252 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:13.996269 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:13.996274 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:13.996353 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:13.996445 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 05:29:13.996485 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:29:13.996658 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40451
--builtin_ntp_servers=127.24.66.84:34581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 05:29:13.996819 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:13.991122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40451 (local address 127.24.66.65:40623)
0812 05:29:13.991369 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:13.991407 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:13.991992 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:13.992300 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:13.992304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:13.992313 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:13.992911 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.992917 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.993747 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.993748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.994700 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:13.994703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:13.995160 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:13.995161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:13.995345 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:13.996042 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:29:13.996042 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:13.996076 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:29:13.996113 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:13.996356 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:13.996363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:13.996367 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:29:13.996439 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 05:29:13.996557 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":109,"threads_started":1}
I20260812 05:29:13.997314 11146 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40451
I20260812 05:29:13.997413 11146 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:13.997639 11146 heartbeater.cc:511] Master 127.24.66.126:40451 requested a full tablet report, sending...
I20260812 05:29:13.998198 10945 ts_manager.cc:194] Registered new tserver with Master: 392c49f0e0da4c64aa73799ed1cf5a24 (127.24.66.65:45649)
I20260812 05:29:13.998922 10945 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.65:40623
W20260812 05:29:14.083954 11149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:14.084172 11149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:14.084201 11149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:14.085402 11149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:14.085443 11149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:14.085470 11149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:29:14.086902 11149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:14.087162 11149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:14.087482 11149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:29:14.093081 11149 server_base.cc:1034] running on GCE node
W20260812 05:29:14.092969 11155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:14.092969 11157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:14.092962 11154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:14.093714 11149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:14.093946 11149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:14.095077 11149 hybrid_clock.cc:648] HybridClock initialized: now 1786512554095037 us; error 56 us; skew 500 ppm
I20260812 05:29:14.096448 11149 webserver.cc:460] Webserver started at http://127.24.66.66:43709/ using document root <none> and password file <none>
I20260812 05:29:14.096741 11149 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:14.096793 11149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:14.097002 11149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:14.098130 11149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "3b7d9bba00844e4fbc83e001f04a1c33"
format_stamp: "Formatted at 2026-08-12 05:29:14 on dist-test-slave-zpfg"
server_key: "9ee61fc80627a22baa1c2f316de67976"
server_key_iv: "dd92ed1d121957b669b25719ae99bc0a"
server_key_version: "encryptionkey@0"
I20260812 05:29:14.098517 11149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "3b7d9bba00844e4fbc83e001f04a1c33"
format_stamp: "Formatted at 2026-08-12 05:29:14 on dist-test-slave-zpfg"
server_key: "9ee61fc80627a22baa1c2f316de67976"
server_key_iv: "dd92ed1d121957b669b25719ae99bc0a"
server_key_version: "encryptionkey@0"
I20260812 05:29:14.099812 11149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:14.100597 11163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:14.100843 11149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:14.100951 11149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "3b7d9bba00844e4fbc83e001f04a1c33"
format_stamp: "Formatted at 2026-08-12 05:29:14 on dist-test-slave-zpfg"
server_key: "9ee61fc80627a22baa1c2f316de67976"
server_key_iv: "dd92ed1d121957b669b25719ae99bc0a"
server_key_version: "encryptionkey@0"
I20260812 05:29:14.101028 11149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:14.120103 11149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:14.120465 11149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:14.120638 11149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:14.120906 11149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:14.121340 11149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:14.121399 11149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:14.121436 11149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:14.121467 11149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:14.127710 11149 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:45433
I20260812 05:29:14.127754 11282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:45433 every 8 connection(s)
I20260812 05:29:14.128108 11149 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:29:14.133172 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 11149
I20260812 05:29:14.133298 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:29:14.133523 24841 external_mini_cluster.cc:1428] Setting key b4cc35e22c0d88018036051b47cc535c
I20260812 05:29:14.134116 11007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.128870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:44409 (local address 127.24.66.126:40451)
0812 05:29:14.128920 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:14.128922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:14.129183 (+ 261us) server_negotiation.cc:407] Connection header received
0812 05:29:14.129732 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:14.129734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:14.129750 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:14.129816 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:14.130613 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.131119 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.132146 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.132357 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.132592 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.133735 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:14.133737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:14.133737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:14.133743 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:14.133760 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:14.133766 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:14.133840 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:14.134027 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 05:29:14.134056 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 05:29:14.134400 11285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.128849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40451 (local address 127.24.66.66:44409)
0812 05:29:14.129042 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:14.129076 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:14.129615 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:14.129899 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:14.129904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:14.129916 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:14.130518 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.130524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.131221 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.131222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.132038 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.132040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.132626 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.132628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.132813 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.133486 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:29:14.133486 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:14.133520 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:29:14.133545 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:14.133855 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:14.133862 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:14.133866 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:29:14.133967 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 05:29:14.134088 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260812 05:29:14.134928 11283 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40451
I20260812 05:29:14.135018 11283 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:14.135048 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40451
--builtin_ntp_servers=127.24.66.84:34581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 05:29:14.135370 11283 heartbeater.cc:511] Master 127.24.66.126:40451 requested a full tablet report, sending...
I20260812 05:29:14.135809 10945 ts_manager.cc:194] Registered new tserver with Master: 3b7d9bba00844e4fbc83e001f04a1c33 (127.24.66.66:45433)
I20260812 05:29:14.136253 10945 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:44409
W20260812 05:29:14.221210 11286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:14.221428 11286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:14.221450 11286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:14.222674 11286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:14.222718 11286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:14.222745 11286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:29:14.224107 11286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:14.224375 11286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:14.224710 11286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:14.230218 11294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:14.230232 11291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:14.230290 11286 server_base.cc:1034] running on GCE node
W20260812 05:29:14.230239 11292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:14.230801 11286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:14.231097 11286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:14.232434 11286 hybrid_clock.cc:648] HybridClock initialized: now 1786512554232402 us; error 42 us; skew 500 ppm
I20260812 05:29:14.233777 11286 webserver.cc:460] Webserver started at http://127.24.66.67:36357/ using document root <none> and password file <none>
I20260812 05:29:14.234006 11286 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:14.234050 11286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:14.234249 11286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:14.235424 11286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "bcd2b94f222943d68b5911dfdaa7a50b"
format_stamp: "Formatted at 2026-08-12 05:29:14 on dist-test-slave-zpfg"
server_key: "fa66dcd49329f862038b5a5f0302c25e"
server_key_iv: "5789f02e5e074c32cea67b762433f156"
server_key_version: "encryptionkey@0"
I20260812 05:29:14.235751 11286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "bcd2b94f222943d68b5911dfdaa7a50b"
format_stamp: "Formatted at 2026-08-12 05:29:14 on dist-test-slave-zpfg"
server_key: "fa66dcd49329f862038b5a5f0302c25e"
server_key_iv: "5789f02e5e074c32cea67b762433f156"
server_key_version: "encryptionkey@0"
I20260812 05:29:14.237228 11286 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:29:14.238200 11300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:14.238411 11286 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:14.238507 11286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "bcd2b94f222943d68b5911dfdaa7a50b"
format_stamp: "Formatted at 2026-08-12 05:29:14 on dist-test-slave-zpfg"
server_key: "fa66dcd49329f862038b5a5f0302c25e"
server_key_iv: "5789f02e5e074c32cea67b762433f156"
server_key_version: "encryptionkey@0"
I20260812 05:29:14.238584 11286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:14.252008 11286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:14.252351 11286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:14.252511 11286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:14.252771 11286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:14.253154 11286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:14.253218 11286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:14.253264 11286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:14.253289 11286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:14.259886 11286 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:44619
I20260812 05:29:14.260076 11419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:44619 every 8 connection(s)
I20260812 05:29:14.260365 11286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:29:14.261577 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 11286
I20260812 05:29:14.261739 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:29:14.261983 24841 external_mini_cluster.cc:1428] Setting key d04cf6feb903d24829a170752928e874
I20260812 05:29:14.267580 11007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.262083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:46887 (local address 127.24.66.126:40451)
0812 05:29:14.262139 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:14.262142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:14.262524 (+ 382us) server_negotiation.cc:407] Connection header received
0812 05:29:14.263340 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:14.263341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:14.263352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:14.263417 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:14.264376 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.264895 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.265973 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.266168 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.266231 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.267353 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:14.267354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:14.267354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:14.267358 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:14.267373 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:14.267377 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:14.267424 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:14.267488 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 05:29:14.267512 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:29:14.267729 11422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.262085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:40451 (local address 127.24.66.67:46887)
0812 05:29:14.262397 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:14.262441 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:14.263119 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:14.263579 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:14.263583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:14.263674 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:14.264227 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.264233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.264985 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.264986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.265861 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.265864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.266316 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.266317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.266543 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.267199 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:29:14.267199 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:14.267230 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:29:14.267261 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:14.267424 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:14.267426 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:14.267429 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:29:14.267477 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 05:29:14.267559 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":191,"threads_started":1}
I20260812 05:29:14.268368 11420 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40451
I20260812 05:29:14.268460 11420 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:14.268612 11420 heartbeater.cc:511] Master 127.24.66.126:40451 requested a full tablet report, sending...
I20260812 05:29:14.268980 10945 ts_manager.cc:194] Registered new tserver with Master: bcd2b94f222943d68b5911dfdaa7a50b (127.24.66.67:44619)
I20260812 05:29:14.269431 10945 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:46887
I20260812 05:29:14.274256 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:29:14.279505 11007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.276239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58260 (local address 127.24.66.126:40451)
0812 05:29:14.276336 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:14.276338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:14.276349 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:29:14.276381 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:14.276382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:14.276390 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:14.276444 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:14.276673 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.277187 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.278125 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.278328 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.278472 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.279163 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:14.279165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:14.279165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:14.279170 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:14.279189 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:14.279194 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:14.279247 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:14.279376 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 05:29:14.279405 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 05:29:14.281368 10945 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:29:14.283154 10945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:29:14.290222 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.285673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:44619 (local address 127.0.0.1:60030)
0812 05:29:14.285862 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:14.285906 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:14.285979 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:14.286506 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:14.286509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:14.286512 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:14.286692 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.286694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.287596 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.287597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.288774 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.288776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.289836 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.289837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.289920 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.289936 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 05:29:14.290051 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 05:29:14.290112 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":98,"thread_start_us":73,"threads_started":1}
I20260812 05:29:14.290230 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.285871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60030 (local address 127.24.66.67:44619)
0812 05:29:14.286248 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:14.286253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:14.286266 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:29:14.286317 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:14.286319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:14.286381 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:14.286571 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:14.286760 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.287501 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.288875 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.289619 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.289840 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.289961 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:14.290078 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 05:29:14.290133 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":92,"threads_started":1}
I20260812 05:29:14.290477 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.285673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:45649 (local address 127.0.0.1:39714)
0812 05:29:14.285882 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:14.285919 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:14.285980 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:14.286506 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:14.286509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:14.286514 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:14.286622 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.286625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.288061 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.288062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.288798 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.288800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.290183 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.290184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.290258 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.290264 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:29:14.290299 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 05:29:14.290378 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":18,"spinlock_wait_cycles":768,"thread_start_us":81,"threads_started":1}
I20260812 05:29:14.290874 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.286084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39714 (local address 127.24.66.65:45649)
0812 05:29:14.286288 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:14.286291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:14.286301 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:29:14.286367 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:14.286369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:14.286383 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:14.286698 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:14.286764 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.287952 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.289121 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.290047 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.290577 (+ 530us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.290693 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:14.290723 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 05:29:14.290777 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
I20260812 05:29:14.291114 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.285725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:45433 (local address 127.0.0.1:53636)
0812 05:29:14.286283 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:14.286295 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:14.286445 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:14.286705 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:14.286706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:14.286708 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:14.286767 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.286769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.288045 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.288046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.289299 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.289302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.290832 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.290833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.290928 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.290933 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:29:14.290996 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 05:29:14.291043 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":31,"thread_start_us":110,"threads_started":1}
I20260812 05:29:14.291123 11352 tablet_service.cc:1467] Processing CreateTablet for tablet 616e60a9fc884a5f89d30bc0e9da6be1 (DEFAULT_TABLE table=test-table [id=ad848422c425491fa0208128dba2c418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:14.291429 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.285985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.24.66.66:45433)
0812 05:29:14.286191 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:14.286197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:14.286412 (+ 215us) server_negotiation.cc:407] Connection header received
0812 05:29:14.286545 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:14.286547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:14.286563 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:14.286634 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:14.286860 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.287953 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.289384 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.290139 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.290200 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.290285 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:14.291296 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0812 05:29:14.291348 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":107,"threads_started":1}
I20260812 05:29:14.291584 11352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 616e60a9fc884a5f89d30bc0e9da6be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:14.291774 11075 tablet_service.cc:1467] Processing CreateTablet for tablet 616e60a9fc884a5f89d30bc0e9da6be1 (DEFAULT_TABLE table=test-table [id=ad848422c425491fa0208128dba2c418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:14.292120 11075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 616e60a9fc884a5f89d30bc0e9da6be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:14.292125 11213 tablet_service.cc:1467] Processing CreateTablet for tablet 616e60a9fc884a5f89d30bc0e9da6be1 (DEFAULT_TABLE table=test-table [id=ad848422c425491fa0208128dba2c418]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:14.292476 11213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 616e60a9fc884a5f89d30bc0e9da6be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:14.295199 11434 tablet_bootstrap.cc:492] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24: Bootstrap starting.
I20260812 05:29:14.295864 11436 tablet_bootstrap.cc:492] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: Bootstrap starting.
I20260812 05:29:14.295984 11435 tablet_bootstrap.cc:492] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: Bootstrap starting.
I20260812 05:29:14.296030 11434 tablet_bootstrap.cc:654] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:14.296499 11434 log.cc:824] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:14.296609 11435 tablet_bootstrap.cc:654] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:14.296592 11436 tablet_bootstrap.cc:654] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:14.296995 11435 log.cc:824] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:14.297009 11436 log.cc:824] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:14.297479 11434 tablet_bootstrap.cc:492] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24: No bootstrap required, opened a new log
I20260812 05:29:14.297572 11434 ts_tablet_manager.cc:1397] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:29:14.297845 11436 tablet_bootstrap.cc:492] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: No bootstrap required, opened a new log
I20260812 05:29:14.297940 11436 ts_tablet_manager.cc:1397] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:29:14.298197 11435 tablet_bootstrap.cc:492] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: No bootstrap required, opened a new log
I20260812 05:29:14.298259 11435 ts_tablet_manager.cc:1397] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:29:14.299969 11434 raft_consensus.cc:348] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 44619 } } peers { permanent_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45649 } } peers { permanent_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45433 } }
I20260812 05:29:14.300173 11434 raft_consensus.cc:374] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:14.300225 11434 raft_consensus.cc:729] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 392c49f0e0da4c64aa73799ed1cf5a24, State: Initialized, Role: FOLLOWER
I20260812 05:29:14.300376 11434 consensus_queue.cc:260] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 44619 } } peers { permanent_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45649 } } peers { permanent_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45433 } }
I20260812 05:29:14.300727 11146 heartbeater.cc:503] Master 127.24.66.126:40451 was elected leader, sending a full tablet report...
I20260812 05:29:14.300849 11436 raft_consensus.cc:348] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 44619 } } peers { permanent_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45649 } } peers { permanent_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45433 } }
I20260812 05:29:14.300849 11435 raft_consensus.cc:348] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 44619 } } peers { permanent_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45649 } } peers { permanent_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45433 } }
I20260812 05:29:14.300711 11434 ts_tablet_manager.cc:1428] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 05:29:14.301008 11435 raft_consensus.cc:374] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:14.301008 11436 raft_consensus.cc:374] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:14.301043 11436 raft_consensus.cc:729] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7d9bba00844e4fbc83e001f04a1c33, State: Initialized, Role: FOLLOWER
I20260812 05:29:14.301180 11436 consensus_queue.cc:260] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 44619 } } peers { permanent_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45649 } } peers { permanent_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45433 } }
I20260812 05:29:14.301476 11436 ts_tablet_manager.cc:1428] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 05:29:14.301566 11283 heartbeater.cc:503] Master 127.24.66.126:40451 was elected leader, sending a full tablet report...
I20260812 05:29:14.301043 11435 raft_consensus.cc:729] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcd2b94f222943d68b5911dfdaa7a50b, State: Initialized, Role: FOLLOWER
I20260812 05:29:14.301801 11435 consensus_queue.cc:260] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 44619 } } peers { permanent_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45649 } } peers { permanent_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45433 } }
I20260812 05:29:14.302084 11435 ts_tablet_manager.cc:1428] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 05:29:14.302104 11420 heartbeater.cc:503] Master 127.24.66.126:40451 was elected leader, sending a full tablet report...
W20260812 05:29:14.378779 11284 tablet.cc:2369] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 05:29:14.491159 11147 tablet.cc:2369] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:29:14.494206 11440 raft_consensus.cc:482] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:29:14.494402 11440 raft_consensus.cc:504] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 44619 } } peers { permanent_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45649 } } peers { permanent_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45433 } }
I20260812 05:29:14.494804 11440 leader_election.cc:288] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcd2b94f222943d68b5911dfdaa7a50b (127.24.66.67:44619), 3b7d9bba00844e4fbc83e001f04a1c33 (127.24.66.66:45433)
I20260812 05:29:14.499244 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.495125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:44705 (local address 127.24.66.66:45433)
0812 05:29:14.495215 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:14.495218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:14.495286 (+ 68us) server_negotiation.cc:407] Connection header received
0812 05:29:14.495331 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:14.495331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:14.495344 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:14.495402 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:14.495762 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.496575 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.498227 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.498814 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.498898 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.498907 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:14.499124 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 05:29:14.499169 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:29:14.499244 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.495188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:42119 (local address 127.24.66.67:44619)
0812 05:29:14.495253 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:14.495274 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:14.495283 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:29:14.495317 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:14.495317 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:14.495329 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:14.495402 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:14.495762 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.496521 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.498227 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.498813 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.498897 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.498907 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:14.499124 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 05:29:14.499169 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 05:29:14.499259 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.495034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:45433 (local address 127.24.66.65:44705)
0812 05:29:14.495187 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:14.495198 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:14.495277 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:14.495501 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:14.495502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:14.495505 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:14.495632 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.495634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.496660 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.496661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.498101 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.498105 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.498933 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.498934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.499029 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.499035 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:29:14.499107 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 05:29:14.499163 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":108,"thread_start_us":71,"threads_started":1}
I20260812 05:29:14.499576 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.495034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:44619 (local address 127.24.66.65:42119)
0812 05:29:14.495117 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:14.495133 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:14.495207 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:14.495501 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:14.495502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:14.495505 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:14.495632 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.495634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.496656 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.496657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.498105 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:14.498106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.498933 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.498934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:14.499039 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.499041 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:29:14.499107 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 05:29:14.499260 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":130}
I20260812 05:29:14.499750 11236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616e60a9fc884a5f89d30bc0e9da6be1" candidate_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" is_pre_election: true
I20260812 05:29:14.499938 11236 raft_consensus.cc:2454] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 392c49f0e0da4c64aa73799ed1cf5a24 in term 0.
I20260812 05:29:14.500202 11028 leader_election.cc:302] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 392c49f0e0da4c64aa73799ed1cf5a24, 3b7d9bba00844e4fbc83e001f04a1c33; no voters:
I20260812 05:29:14.500295 11373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616e60a9fc884a5f89d30bc0e9da6be1" candidate_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" is_pre_election: true
I20260812 05:29:14.500381 11440 raft_consensus.cc:2781] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:29:14.500440 11440 raft_consensus.cc:482] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:29:14.500475 11440 raft_consensus.cc:3037] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:14.500464 11373 raft_consensus.cc:2454] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 392c49f0e0da4c64aa73799ed1cf5a24 in term 0.
I20260812 05:29:14.501358 11440 raft_consensus.cc:504] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 44619 } } peers { permanent_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45649 } } peers { permanent_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45433 } }
I20260812 05:29:14.501526 11440 leader_election.cc:288] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [CANDIDATE]: Term 1 election: Requested vote from peers bcd2b94f222943d68b5911dfdaa7a50b (127.24.66.67:44619), 3b7d9bba00844e4fbc83e001f04a1c33 (127.24.66.66:45433)
I20260812 05:29:14.501713 11373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616e60a9fc884a5f89d30bc0e9da6be1" candidate_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcd2b94f222943d68b5911dfdaa7a50b"
I20260812 05:29:14.501777 11236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616e60a9fc884a5f89d30bc0e9da6be1" candidate_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b7d9bba00844e4fbc83e001f04a1c33"
I20260812 05:29:14.501829 11373 raft_consensus.cc:3037] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:14.501852 11236 raft_consensus.cc:3037] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:14.502821 11236 raft_consensus.cc:2454] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 392c49f0e0da4c64aa73799ed1cf5a24 in term 1.
I20260812 05:29:14.502915 11373 raft_consensus.cc:2454] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 392c49f0e0da4c64aa73799ed1cf5a24 in term 1.
I20260812 05:29:14.503041 11028 leader_election.cc:302] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 392c49f0e0da4c64aa73799ed1cf5a24, 3b7d9bba00844e4fbc83e001f04a1c33; no voters:
I20260812 05:29:14.503159 11440 raft_consensus.cc:2781] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:14.503366 11440 raft_consensus.cc:686] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 1 LEADER]: Becoming Leader. State: Replica: 392c49f0e0da4c64aa73799ed1cf5a24, State: Running, Role: LEADER
I20260812 05:29:14.503495 11440 consensus_queue.cc:237] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 44619 } } peers { permanent_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45649 } } peers { permanent_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45433 } }
I20260812 05:29:14.504405 10945 catalog_manager.cc:5495] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 392c49f0e0da4c64aa73799ed1cf5a24 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 44619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "392c49f0e0da4c64aa73799ed1cf5a24" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45433 } health_report { overall_health: UNKNOWN } } }
W20260812 05:29:14.510972 11421 tablet.cc:2369] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:29:14.519059 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.516582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.24.66.65:45649)
0812 05:29:14.516665 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:14.516671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:14.516681 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:29:14.516718 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:14.516718 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:14.516739 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:14.516808 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:14.517007 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.517480 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.518270 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:14.518519 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:14.518576 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:14.518760 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:14.518866 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:14.518944 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:14.518961 (+ 17us) server_negotiation.cc:299] Negotiation successful
0812 05:29:14.518980 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 05:29:14.521184 11236 raft_consensus.cc:1261] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33 [term 1 FOLLOWER]: Refusing update from remote peer 392c49f0e0da4c64aa73799ed1cf5a24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:14.521215 11373 raft_consensus.cc:1261] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b [term 1 FOLLOWER]: Refusing update from remote peer 392c49f0e0da4c64aa73799ed1cf5a24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:14.521555 11440 consensus_queue.cc:1035] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b7d9bba00844e4fbc83e001f04a1c33" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 45433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:14.521701 11444 consensus_queue.cc:1035] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcd2b94f222943d68b5911dfdaa7a50b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 44619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:14.525481 11448 mvcc.cc:205] Tried to move back new op lower bound from 7317555423315718144 to 7317555423246991360. Current Snapshot: MvccSnapshot[applied={T|T < 7317555423315718144}]
I20260812 05:29:14.531967 10945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:14.532089 10945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:14.533479 10945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a172761b37a4d5cbcc0bdd5e9f87669: Sending DeleteTablet for 3 replicas of tablet 616e60a9fc884a5f89d30bc0e9da6be1
I20260812 05:29:14.533896 11449 mvcc.cc:205] Tried to move back new op lower bound from 7317555423315718144 to 7317555423246991360. Current Snapshot: MvccSnapshot[applied={T|T < 7317555423315718144}]
I20260812 05:29:14.534020 11075 tablet_service.cc:1514] Processing DeleteTablet for tablet 616e60a9fc884a5f89d30bc0e9da6be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:14 UTC) from {username='slave'} at 127.0.0.1:39714
I20260812 05:29:14.534277 11213 tablet_service.cc:1514] Processing DeleteTablet for tablet 616e60a9fc884a5f89d30bc0e9da6be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:14 UTC) from {username='slave'} at 127.0.0.1:53636
I20260812 05:29:14.534569 11456 tablet_replica.cc:330] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24: stopping tablet replica
I20260812 05:29:14.534586 11457 tablet_replica.cc:330] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: stopping tablet replica
I20260812 05:29:14.534667 11456 raft_consensus.cc:2229] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:29:14.534715 11457 raft_consensus.cc:2229] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:29:14.534770 11456 raft_consensus.cc:2258] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:14.534824 11457 raft_consensus.cc:2258] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:14.535311 11457 ts_tablet_manager.cc:1905] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:14.535346 11456 ts_tablet_manager.cc:1905] T 616e60a9fc884a5f89d30bc0e9da6be1 P 392c49f0e0da4c64aa73799ed1cf5a24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:14.535449 11352 tablet_service.cc:1514] Processing DeleteTablet for tablet 616e60a9fc884a5f89d30bc0e9da6be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:14 UTC) from {username='slave'} at 127.0.0.1:60030
I20260812 05:29:14.535670 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 11010
I20260812 05:29:14.538398 11457 ts_tablet_manager.cc:1918] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:14.538475 11457 log.cc:1196] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/616e60a9fc884a5f89d30bc0e9da6be1
I20260812 05:29:14.538765 11457 ts_tablet_manager.cc:1939] T 616e60a9fc884a5f89d30bc0e9da6be1 P 3b7d9bba00844e4fbc83e001f04a1c33: Deleting consensus metadata
I20260812 05:29:14.539141 11458 tablet_replica.cc:330] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: stopping tablet replica
I20260812 05:29:14.539264 11458 raft_consensus.cc:2229] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:29:14.539388 11458 raft_consensus.cc:2258] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:14.539845 10931 catalog_manager.cc:4841] TS 3b7d9bba00844e4fbc83e001f04a1c33 (127.24.66.66:45433): tablet 616e60a9fc884a5f89d30bc0e9da6be1 (table test-table [id=ad848422c425491fa0208128dba2c418]) successfully deleted
I20260812 05:29:14.542089 11458 ts_tablet_manager.cc:1905] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:14.544127 11458 ts_tablet_manager.cc:1918] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:14.544220 11458 log.cc:1196] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/616e60a9fc884a5f89d30bc0e9da6be1
I20260812 05:29:14.544495 11458 ts_tablet_manager.cc:1939] T 616e60a9fc884a5f89d30bc0e9da6be1 P bcd2b94f222943d68b5911dfdaa7a50b: Deleting consensus metadata
W20260812 05:29:14.544883 10929 connection.cc:535] server connection from 127.24.66.65:40623 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 05:29:14.544909 10932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:45649 (error 108)
I20260812 05:29:14.545001 10929 catalog_manager.cc:4841] TS bcd2b94f222943d68b5911dfdaa7a50b (127.24.66.67:44619): tablet 616e60a9fc884a5f89d30bc0e9da6be1 (table test-table [id=ad848422c425491fa0208128dba2c418]) successfully deleted
I20260812 05:29:14.545462 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:14.545306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:45649 (local address 127.0.0.1:39728)
0812 05:29:14.545370 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:14.545421 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:45649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 05:29:14.545575 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 11149
W20260812 05:29:14.545615 10932 catalog_manager.cc:4584] TS 392c49f0e0da4c64aa73799ed1cf5a24 (127.24.66.65:45649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 616e60a9fc884a5f89d30bc0e9da6be1: Network error: Client connection negotiation failed: client connection to 127.24.66.65:45649: connect: Connection refused (error 111)
I20260812 05:29:14.552320 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 11286
I20260812 05:29:14.560029 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 10914
2026-08-12T05:29:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (926 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:14 dist-test-slave-zpfg krb5kdc[11462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:14 dist-test-slave-zpfg krb5kdc[11462](info): set up 2 sockets
Aug 12 05:29:14 dist-test-slave-zpfg krb5kdc[11462](info): commencing operation
krb5kdc: starting...
W20260812 05:29:15.743585 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:15 dist-test-slave-zpfg krb5kdc[11462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:29:15.878630 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:45443
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:33873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:45443
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:15.963330 11475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:15.963599 11475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:15.963644 11475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:15.964850 11475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:15.964931 11475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:15.964954 11475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:15.964974 11475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:15.965006 11475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:15.966606 11475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:33873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:45443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:45443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:15.966928 11475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:15.967233 11475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:15.972743 11482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:15.972898 11481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:15.972927 11484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:15.973196 11475 server_base.cc:1034] running on GCE node
I20260812 05:29:15.973500 11475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:15.973866 11475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:15.975041 11475 hybrid_clock.cc:648] HybridClock initialized: now 1786512555975027 us; error 49 us; skew 500 ppm
Aug 12 05:29:15 dist-test-slave-zpfg krb5kdc[11462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512555, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:15.977706 11475 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:29:15.978204 11475 webserver.cc:460] Webserver started at http://127.24.66.126:36161/ using document root <none> and password file <none>
I20260812 05:29:15.978504 11475 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:15.978575 11475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:15.978739 11475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:15.979856 11475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "00956b3d9d9f4a48966d275d345eeb2d"
format_stamp: "Formatted at 2026-08-12 05:29:15 on dist-test-slave-zpfg"
server_key: "941c0025cef54a3e219369c0815044e7"
server_key_iv: "6efb215e254a37a47a91cb3dfc2969dc"
server_key_version: "encryptionkey@0"
I20260812 05:29:15.980243 11475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "00956b3d9d9f4a48966d275d345eeb2d"
format_stamp: "Formatted at 2026-08-12 05:29:15 on dist-test-slave-zpfg"
server_key: "941c0025cef54a3e219369c0815044e7"
server_key_iv: "6efb215e254a37a47a91cb3dfc2969dc"
server_key_version: "encryptionkey@0"
I20260812 05:29:15.981719 11475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:15.982527 11493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:15.982693 11475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:15.982836 11475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "00956b3d9d9f4a48966d275d345eeb2d"
format_stamp: "Formatted at 2026-08-12 05:29:15 on dist-test-slave-zpfg"
server_key: "941c0025cef54a3e219369c0815044e7"
server_key_iv: "6efb215e254a37a47a91cb3dfc2969dc"
server_key_version: "encryptionkey@0"
I20260812 05:29:15.982918 11475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:15.993383 11475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:15.996057 11475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:15.996270 11475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:16.000931 11475 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:45443
I20260812 05:29:16.000976 11546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:45443 every 8 connection(s)
I20260812 05:29:16.001361 11475 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:16.002254 11547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:16.004727 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 11475
I20260812 05:29:16.004841 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:16.004951 11547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d: Bootstrap starting.
I20260812 05:29:16.005084 24841 external_mini_cluster.cc:1428] Setting key be362a0fe4df60140bb943eaab7a6ecd
I20260812 05:29:16.005697 11547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:16.006054 11547 log.cc:824] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:16.006826 11547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d: No bootstrap required, opened a new log
I20260812 05:29:16.008781 11547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00956b3d9d9f4a48966d275d345eeb2d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45443 } }
I20260812 05:29:16.008955 11547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:16.009006 11547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00956b3d9d9f4a48966d275d345eeb2d, State: Initialized, Role: FOLLOWER
I20260812 05:29:16.009151 11547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00956b3d9d9f4a48966d275d345eeb2d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45443 } }
I20260812 05:29:16.009234 11547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:16.009279 11547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:16.009328 11547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:16.009928 11547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00956b3d9d9f4a48966d275d345eeb2d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45443 } }
I20260812 05:29:16.010067 11547 leader_election.cc:302] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00956b3d9d9f4a48966d275d345eeb2d; no voters:
I20260812 05:29:16.010280 11547 leader_election.cc:288] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:16.010305 11552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:16.010449 11552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [term 1 LEADER]: Becoming Leader. State: Replica: 00956b3d9d9f4a48966d275d345eeb2d, State: Running, Role: LEADER
I20260812 05:29:16.010540 11552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00956b3d9d9f4a48966d275d345eeb2d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45443 } }
I20260812 05:29:16.010601 11547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:16.010915 11554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00956b3d9d9f4a48966d275d345eeb2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00956b3d9d9f4a48966d275d345eeb2d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45443 } } }
I20260812 05:29:16.011018 11554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:16.010907 11553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00956b3d9d9f4a48966d275d345eeb2d. Latest consensus state: current_term: 1 leader_uuid: "00956b3d9d9f4a48966d275d345eeb2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00956b3d9d9f4a48966d275d345eeb2d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45443 } } }
I20260812 05:29:16.011714 11553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:16.012059 11558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 05:29:16 dist-test-slave-zpfg krb5kdc[11462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:16.012879 11558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:16.015000 11558 catalog_manager.cc:1347] Generated new cluster ID: b24b08a40ddd425092c082c986f8573b
I20260812 05:29:16.015053 11558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:16.017241 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.005694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47494 (local address 127.24.66.126:45443)
0812 05:29:16.005972 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.005980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.006008 (+ 28us) server_negotiation.cc:407] Connection header received
0812 05:29:16.006628 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.006636 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.006697 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.006858 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:16.007564 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.008469 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.009439 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.009879 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.010013 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.013246 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:16.013250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:16.013250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:16.013262 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:16.015505 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:16.015961 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:16.015964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:16.015964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:16.016068 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:16.016313 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:16.016314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:16.016314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:16.016604 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:16.016760 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.016886 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.017019 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":89,"threads_started":1}
I20260812 05:29:16.032804 11558 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:16.033403 11558 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:16.040973 11558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d: Generated new TSK 0
I20260812 05:29:16.041221 11558 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:29:16.077077 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45443
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:33873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:16.163834 11573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:16.164052 11573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:16.164076 11573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:16.165275 11573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:16.165324 11573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:16.165351 11573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:29:16.166958 11573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:33873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:16.167223 11573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:16.167557 11573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:16.173185 11580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:16.173163 11582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:16.173322 11573 server_base.cc:1034] running on GCE node
W20260812 05:29:16.173197 11584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:16.173683 11573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:16.173993 11573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:16.175163 11573 hybrid_clock.cc:648] HybridClock initialized: now 1786512556175150 us; error 45 us; skew 500 ppm
Aug 12 05:29:16 dist-test-slave-zpfg krb5kdc[11462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512556, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:16.177799 11573 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:29:16.178298 11573 webserver.cc:460] Webserver started at http://127.24.66.65:37627/ using document root <none> and password file <none>
I20260812 05:29:16.178597 11573 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:16.178669 11573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:16.178817 11573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:16.179937 11573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "60d7e306849c46dda1a853225b0773e3"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "9dd6a97fa93969fdbae27b16370f95d6"
server_key_iv: "b9143cd5178bb626be56e599511b7fab"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.180311 11573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "60d7e306849c46dda1a853225b0773e3"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "9dd6a97fa93969fdbae27b16370f95d6"
server_key_iv: "b9143cd5178bb626be56e599511b7fab"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.181699 11573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:16.182519 11592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:16.182691 11573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:16.182793 11573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "60d7e306849c46dda1a853225b0773e3"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "9dd6a97fa93969fdbae27b16370f95d6"
server_key_iv: "b9143cd5178bb626be56e599511b7fab"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.182879 11573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:16.202001 11573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:16.204272 11573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:16.204450 11573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:16.204710 11573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:16.205054 11573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:16.205108 11573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:16.205143 11573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:16.205185 11573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:16.211856 11573 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:38549
I20260812 05:29:16.211895 11712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:38549 every 8 connection(s)
I20260812 05:29:16.212344 11573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:29:16.213052 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 11573
I20260812 05:29:16.213155 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:29:16.213425 24841 external_mini_cluster.cc:1428] Setting key b7fc8355831343d790c8513c1d25bffc
Aug 12 05:29:16 dist-test-slave-zpfg krb5kdc[11462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512556, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:16.223753 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.213116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45443 (local address 127.24.66.65:36223)
0812 05:29:16.213284 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:16.213330 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:16.213995 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:16.214273 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:16.214278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:16.214950 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:16.215181 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.215186 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.216241 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.216243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.217587 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.217590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.217988 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.217989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.218141 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.218732 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:16.218745 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:16.220420 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:16.222280 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:16.222287 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:16.222289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:16.222574 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:16.222789 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:16.222790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:16.222790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:16.222888 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:16.223197 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:16.223202 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:16.223348 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 05:29:16.223428 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 05:29:16.223577 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260812 05:29:16.223747 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.213166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:36223 (local address 127.24.66.126:45443)
0812 05:29:16.213243 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.213283 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.213428 (+ 145us) server_negotiation.cc:407] Connection header received
0812 05:29:16.214111 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.214112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.214125 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.214188 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:16.215310 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.216113 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.217683 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.217889 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.217946 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.220522 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:16.220525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:16.220525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:16.220534 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:16.222168 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:16.222650 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:16.222653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:16.222653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:16.222717 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:16.222970 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:16.222971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:16.222972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:16.223126 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:16.223207 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.223550 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.223647 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
I20260812 05:29:16.224277 11713 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45443
I20260812 05:29:16.224395 11713 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:16.224584 11713 heartbeater.cc:511] Master 127.24.66.126:45443 requested a full tablet report, sending...
I20260812 05:29:16.225173 11510 ts_manager.cc:194] Registered new tserver with Master: 60d7e306849c46dda1a853225b0773e3 (127.24.66.65:38549)
I20260812 05:29:16.225766 11510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:36223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:29:16.241322 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45443
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:33873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:16.328899 11719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:16.329111 11719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:16.329135 11719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:16.330369 11719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:16.330413 11719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:16.330441 11719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:29:16.331962 11719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:33873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:16.332228 11719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:16.332484 11719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:16.337872 11726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:16.338001 11719 server_base.cc:1034] running on GCE node
W20260812 05:29:16.337841 11725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:16.337889 11728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:16.338315 11719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:16.338634 11719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:16.339807 11719 hybrid_clock.cc:648] HybridClock initialized: now 1786512556339802 us; error 35 us; skew 500 ppm
Aug 12 05:29:16 dist-test-slave-zpfg krb5kdc[11462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512556, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:16.342517 11719 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:29:16.343019 11719 webserver.cc:460] Webserver started at http://127.24.66.66:34525/ using document root <none> and password file <none>
I20260812 05:29:16.343264 11719 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:16.343335 11719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:16.343477 11719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:16.344578 11719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "fafa9ace267902d34cdfa6c7770122ac"
server_key_iv: "731e6205d415286b4bcdea7db4fa4677"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.344964 11719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "fafa9ace267902d34cdfa6c7770122ac"
server_key_iv: "731e6205d415286b4bcdea7db4fa4677"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.346828 11719 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 05:29:16.347872 11735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:16.348086 11719 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:16.348205 11719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "fafa9ace267902d34cdfa6c7770122ac"
server_key_iv: "731e6205d415286b4bcdea7db4fa4677"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.348295 11719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:16.369201 11719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:16.371471 11719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:16.371665 11719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:16.371937 11719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:16.372273 11719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:16.372328 11719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:16.372375 11719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:16.372409 11719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:16.378868 11719 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:41135
I20260812 05:29:16.378911 11857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:41135 every 8 connection(s)
I20260812 05:29:16.379273 11719 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
Aug 12 05:29:16 dist-test-slave-zpfg krb5kdc[11462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512556, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:16.388067 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 11719
I20260812 05:29:16.388206 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:29:16.388451 24841 external_mini_cluster.cc:1428] Setting key d0d0b0e40c5328f966f58ced5d2b0886
I20260812 05:29:16.389684 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.380089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:39815 (local address 127.24.66.126:45443)
0812 05:29:16.380138 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.380140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.380371 (+ 231us) server_negotiation.cc:407] Connection header received
0812 05:29:16.380835 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.380836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.380848 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.380917 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:16.381876 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.382483 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.383506 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.383718 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.383781 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.386157 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:16.386158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:16.386159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:16.386166 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:16.387746 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:16.388379 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:16.388380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:16.388381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:16.388437 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:16.388757 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:16.388758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:16.388758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:16.388955 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:16.389028 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.389539 (+ 511us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.389616 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:29:16.390045 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.380045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45443 (local address 127.24.66.66:39815)
0812 05:29:16.380226 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:16.380271 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:16.380747 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:16.380974 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:16.380978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:16.381571 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:16.381772 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.381777 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.382595 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.382596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.383424 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.383427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.383802 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.383803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.383907 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.384408 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:16.384416 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:16.386034 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:16.387868 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:16.387878 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:16.387881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:16.388279 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:16.388510 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:16.388511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:16.388511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:16.388679 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:16.389030 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:16.389036 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:16.389375 (+ 339us) client_negotiation.cc:769] Sending connection context
0812 05:29:16.389515 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 05:29:16.389729 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260812 05:29:16.390661 11858 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45443
I20260812 05:29:16.390780 11858 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:16.391002 11858 heartbeater.cc:511] Master 127.24.66.126:45443 requested a full tablet report, sending...
I20260812 05:29:16.391443 11510 ts_manager.cc:194] Registered new tserver with Master: b5bafe3ecb6e4d758d040b8c9596f1fc (127.24.66.66:41135)
I20260812 05:29:16.391824 11510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:39815
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:29:16.418005 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45443
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:33873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:16.507849 11863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:16.508085 11863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:16.508122 11863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:16.509418 11863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:16.509469 11863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:16.509524 11863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:29:16.511201 11863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:33873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:16.511456 11863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:16.511755 11863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:16.517081 11869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:16.517056 11868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:16.517329 11863 server_base.cc:1034] running on GCE node
W20260812 05:29:16.517056 11871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:16.517827 11863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:16.518082 11863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:16.519246 11863 hybrid_clock.cc:648] HybridClock initialized: now 1786512556519233 us; error 34 us; skew 500 ppm
Aug 12 05:29:16 dist-test-slave-zpfg krb5kdc[11462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512556, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:16.521837 11863 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:29:16.522414 11863 webserver.cc:460] Webserver started at http://127.24.66.67:43569/ using document root <none> and password file <none>
I20260812 05:29:16.522655 11863 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:16.522701 11863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:16.522881 11863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:16.523998 11863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "0640c38bd52b469fb522125adca1f371"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "747bb662b755fe065ce621e297823907"
server_key_iv: "f1d3811e310b1f4e7e97737503bb5d48"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.524379 11863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "0640c38bd52b469fb522125adca1f371"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "747bb662b755fe065ce621e297823907"
server_key_iv: "f1d3811e310b1f4e7e97737503bb5d48"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.525799 11863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:16.526748 11878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:16.526979 11863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:16.527096 11863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "0640c38bd52b469fb522125adca1f371"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "747bb662b755fe065ce621e297823907"
server_key_iv: "f1d3811e310b1f4e7e97737503bb5d48"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.527177 11863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:16.543076 11863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:16.545501 11863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:16.545679 11863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:16.545949 11863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:16.546484 11863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:16.546550 11863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:16.546594 11863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:16.546622 11863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:16.553272 11863 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:39081
I20260812 05:29:16.553323 12000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:39081 every 8 connection(s)
I20260812 05:29:16.553699 11863 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:29:16.554112 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 11863
I20260812 05:29:16.554229 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:29:16.554507 24841 external_mini_cluster.cc:1428] Setting key 5e519c489d7fd42c76cc0bc8bda8132d
Aug 12 05:29:16 dist-test-slave-zpfg krb5kdc[11462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512556, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:16.564384 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.554592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:52761 (local address 127.24.66.126:45443)
0812 05:29:16.554739 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.554742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.554983 (+ 241us) server_negotiation.cc:407] Connection header received
0812 05:29:16.555615 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.555616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.555634 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.555690 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:16.556702 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.557219 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.558245 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.558546 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.558633 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.560892 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:16.560893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:16.560893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:16.560901 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:16.562664 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:16.563240 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:16.563241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:16.563242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:16.563301 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:16.563602 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:16.563603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:16.563603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:16.563758 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:16.563849 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.564182 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.564279 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260812 05:29:16.564638 12003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.554589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:45443 (local address 127.24.66.67:52761)
0812 05:29:16.554830 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:16.554880 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:16.555508 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:16.555769 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:16.555773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:16.556388 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:16.556603 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.556608 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.557322 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.557322 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.558151 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.558153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.558634 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.558635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.558798 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.559329 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:16.559337 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:16.560794 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:16.562778 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:16.562787 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:16.562789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:16.563043 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:16.563386 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:16.563387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:16.563387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:16.563514 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:16.563838 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:16.563845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:16.564018 (+ 173us) client_negotiation.cc:769] Sending connection context
0812 05:29:16.564141 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 05:29:16.564331 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":155,"threads_started":1}
I20260812 05:29:16.565428 12001 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45443
I20260812 05:29:16.565544 12001 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:16.565744 12001 heartbeater.cc:511] Master 127.24.66.126:45443 requested a full tablet report, sending...
I20260812 05:29:16.566318 11510 ts_manager.cc:194] Registered new tserver with Master: 0640c38bd52b469fb522125adca1f371 (127.24.66.67:39081)
I20260812 05:29:16.566808 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:29:16.566938 11510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:52761
I20260812 05:29:16.575309 11550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.568525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.24.66.126:45443)
0812 05:29:16.568585 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.568587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.568683 (+ 96us) server_negotiation.cc:407] Connection header received
0812 05:29:16.568725 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.568725 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.568735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.568848 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:16.569512 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.570056 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.570881 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.571084 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.571155 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.572618 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:16.572620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:16.572620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:16.572627 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:16.573977 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:16.574443 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:16.574444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:16.574445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:16.574513 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:16.574727 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:16.574728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:16.574728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:16.574880 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:16.574970 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.575099 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.575171 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 05:29:16.577245 11510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:29:16.579241 11510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:29:16.586490 12013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.581872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:38549 (local address 127.0.0.1:35316)
0812 05:29:16.582039 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:16.582087 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:16.582158 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:16.582681 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:16.582684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:16.582687 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:16.582790 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.582792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.583801 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.583802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.584955 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.584957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.586118 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.586119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.586208 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.586223 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 05:29:16.586304 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 05:29:16.586391 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260812 05:29:16.586730 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.581881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:39081 (local address 127.0.0.1:40832)
0812 05:29:16.582434 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:16.582447 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:16.582505 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:16.582761 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:16.582762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:16.582765 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:16.582816 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.582818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.584329 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.584330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.585466 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.585468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.586424 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.586425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.586505 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.586509 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:29:16.586571 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 05:29:16.586628 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":45,"thread_start_us":98,"threads_started":1}
I20260812 05:29:16.586853 12016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.582203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.24.66.67:39081)
0812 05:29:16.582481 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.582486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.582500 (+ 14us) server_negotiation.cc:407] Connection header received
0812 05:29:16.582605 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.582607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.582626 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.582902 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:16.582983 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.584212 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.585553 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.586217 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.586420 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.586484 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.586584 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.586645 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":118,"threads_started":1}
I20260812 05:29:16.587023 12014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.581987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35316 (local address 127.24.66.65:38549)
0812 05:29:16.582480 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.582486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.582497 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:29:16.582542 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.582545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.582561 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.582876 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:16.582956 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.583689 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.585051 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.586026 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.586120 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.586218 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.586758 (+ 540us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.586918 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":109,"threads_started":1}
I20260812 05:29:16.587421 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.582093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:41135 (local address 127.0.0.1:53454)
0812 05:29:16.582609 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:16.582617 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:16.582661 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:16.582941 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:16.582941 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:16.582944 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:16.582978 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.582979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.584462 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.584463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.585807 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.585810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.587173 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.587174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.587249 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.587254 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:29:16.587307 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 05:29:16.587352 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":107,"threads_started":1}
I20260812 05:29:16.587690 12017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.582233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53454 (local address 127.24.66.66:41135)
0812 05:29:16.582481 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.582487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.582720 (+ 233us) server_negotiation.cc:407] Connection header received
0812 05:29:16.582765 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.582767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.582781 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.582853 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:16.583049 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.584342 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.585905 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.587011 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.587078 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.587133 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.587562 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.587608 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":90,"threads_started":1}
I20260812 05:29:16.587682 11931 tablet_service.cc:1467] Processing CreateTablet for tablet 517a37133f214af4a779a429852152f2 (DEFAULT_TABLE table=test-table [id=bb2f0b2c38fb409685e6f105c61e0723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:16.587764 11643 tablet_service.cc:1467] Processing CreateTablet for tablet 517a37133f214af4a779a429852152f2 (DEFAULT_TABLE table=test-table [id=bb2f0b2c38fb409685e6f105c61e0723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:16.588122 11643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 517a37133f214af4a779a429852152f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:16.588117 11931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 517a37133f214af4a779a429852152f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:16.588382 11788 tablet_service.cc:1467] Processing CreateTablet for tablet 517a37133f214af4a779a429852152f2 (DEFAULT_TABLE table=test-table [id=bb2f0b2c38fb409685e6f105c61e0723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:16.588708 11788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 517a37133f214af4a779a429852152f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:16.590986 12019 tablet_bootstrap.cc:492] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3: Bootstrap starting.
I20260812 05:29:16.591650 12019 tablet_bootstrap.cc:654] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:16.592147 12019 log.cc:824] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:16.592379 12020 tablet_bootstrap.cc:492] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371: Bootstrap starting.
I20260812 05:29:16.592860 12022 tablet_bootstrap.cc:492] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: Bootstrap starting.
I20260812 05:29:16.592998 12020 tablet_bootstrap.cc:654] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:16.593616 12020 log.cc:824] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:16.593847 12022 tablet_bootstrap.cc:654] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:16.594465 12022 log.cc:824] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:16.594579 12020 tablet_bootstrap.cc:492] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371: No bootstrap required, opened a new log
I20260812 05:29:16.594610 12019 tablet_bootstrap.cc:492] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3: No bootstrap required, opened a new log
I20260812 05:29:16.594655 12020 ts_tablet_manager.cc:1397] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:29:16.594691 12019 ts_tablet_manager.cc:1397] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 05:29:16.596050 12020 raft_consensus.cc:348] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d7e306849c46dda1a853225b0773e3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38549 } } peers { permanent_uuid: "0640c38bd52b469fb522125adca1f371" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39081 } } peers { permanent_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41135 } }
I20260812 05:29:16.596211 12020 raft_consensus.cc:374] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:16.596235 12020 raft_consensus.cc:729] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0640c38bd52b469fb522125adca1f371, State: Initialized, Role: FOLLOWER
I20260812 05:29:16.596371 12020 consensus_queue.cc:260] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d7e306849c46dda1a853225b0773e3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38549 } } peers { permanent_uuid: "0640c38bd52b469fb522125adca1f371" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39081 } } peers { permanent_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41135 } }
I20260812 05:29:16.596295 12019 raft_consensus.cc:348] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d7e306849c46dda1a853225b0773e3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38549 } } peers { permanent_uuid: "0640c38bd52b469fb522125adca1f371" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39081 } } peers { permanent_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41135 } }
I20260812 05:29:16.596522 12019 raft_consensus.cc:374] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:16.596562 12019 raft_consensus.cc:729] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d7e306849c46dda1a853225b0773e3, State: Initialized, Role: FOLLOWER
I20260812 05:29:16.596665 12020 ts_tablet_manager.cc:1428] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:29:16.596683 12019 consensus_queue.cc:260] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d7e306849c46dda1a853225b0773e3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38549 } } peers { permanent_uuid: "0640c38bd52b469fb522125adca1f371" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39081 } } peers { permanent_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41135 } }
I20260812 05:29:16.596808 12001 heartbeater.cc:503] Master 127.24.66.126:45443 was elected leader, sending a full tablet report...
I20260812 05:29:16.597005 12019 ts_tablet_manager.cc:1428] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 05:29:16.597031 12022 tablet_bootstrap.cc:492] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: No bootstrap required, opened a new log
I20260812 05:29:16.597036 11713 heartbeater.cc:503] Master 127.24.66.126:45443 was elected leader, sending a full tablet report...
I20260812 05:29:16.597095 12022 ts_tablet_manager.cc:1397] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 05:29:16.599205 12022 raft_consensus.cc:348] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d7e306849c46dda1a853225b0773e3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38549 } } peers { permanent_uuid: "0640c38bd52b469fb522125adca1f371" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39081 } } peers { permanent_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41135 } }
I20260812 05:29:16.599345 12022 raft_consensus.cc:374] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:16.599367 12022 raft_consensus.cc:729] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bafe3ecb6e4d758d040b8c9596f1fc, State: Initialized, Role: FOLLOWER
I20260812 05:29:16.599472 12022 consensus_queue.cc:260] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d7e306849c46dda1a853225b0773e3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38549 } } peers { permanent_uuid: "0640c38bd52b469fb522125adca1f371" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39081 } } peers { permanent_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41135 } }
I20260812 05:29:16.599701 12022 ts_tablet_manager.cc:1428] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 05:29:16.599781 11858 heartbeater.cc:503] Master 127.24.66.126:45443 was elected leader, sending a full tablet report...
W20260812 05:29:16.629863 11859 tablet.cc:2369] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:29:16.706282 12026 raft_consensus.cc:482] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:29:16.706487 12026 raft_consensus.cc:504] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d7e306849c46dda1a853225b0773e3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38549 } } peers { permanent_uuid: "0640c38bd52b469fb522125adca1f371" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39081 } } peers { permanent_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41135 } }
I20260812 05:29:16.706830 12026 leader_election.cc:288] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0640c38bd52b469fb522125adca1f371 (127.24.66.67:39081), b5bafe3ecb6e4d758d040b8c9596f1fc (127.24.66.66:41135)
I20260812 05:29:16.710911 12017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.707320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:56197 (local address 127.24.66.66:41135)
0812 05:29:16.707374 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.707376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.707444 (+ 68us) server_negotiation.cc:407] Connection header received
0812 05:29:16.707479 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.707480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.707489 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.707543 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:16.707718 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.708474 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.710090 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.710561 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.710626 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.710635 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.710815 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.710848 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 05:29:16.710943 12028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.707216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:41135 (local address 127.24.66.65:56197)
0812 05:29:16.707375 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:16.707384 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:16.707439 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:16.707541 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:16.707542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:16.707542 (+ 0us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:16.707640 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.707641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.708595 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.708596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.709935 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.709939 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.710650 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.710651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.710735 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.710741 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:29:16.710804 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 05:29:16.710872 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":111,"thread_start_us":93,"threads_started":1}
I20260812 05:29:16.711359 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.707078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:39081 (local address 127.24.66.65:46541)
0812 05:29:16.707135 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:16.707148 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:16.707207 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:16.707531 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:16.707532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:16.707534 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:16.707639 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.707641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.708595 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.708596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.709935 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:16.709939 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.711066 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.711089 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:16.711185 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.711189 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:29:16.711240 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 05:29:16.711286 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":15}
I20260812 05:29:16.711401 12016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.707319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:46541 (local address 127.24.66.67:39081)
0812 05:29:16.707390 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.707392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.707401 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:29:16.707433 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.707434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.707444 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.707512 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:16.707719 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.708521 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.710067 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.710733 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.710815 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.710824 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.711268 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.711322 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 05:29:16.711583 11808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "517a37133f214af4a779a429852152f2" candidate_uuid: "60d7e306849c46dda1a853225b0773e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" is_pre_election: true
I20260812 05:29:16.711773 11808 raft_consensus.cc:2454] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60d7e306849c46dda1a853225b0773e3 in term 0.
I20260812 05:29:16.711792 11952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "517a37133f214af4a779a429852152f2" candidate_uuid: "60d7e306849c46dda1a853225b0773e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0640c38bd52b469fb522125adca1f371" is_pre_election: true
I20260812 05:29:16.711926 11952 raft_consensus.cc:2454] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60d7e306849c46dda1a853225b0773e3 in term 0.
I20260812 05:29:16.712038 11593 leader_election.cc:302] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d7e306849c46dda1a853225b0773e3, b5bafe3ecb6e4d758d040b8c9596f1fc; no voters:
I20260812 05:29:16.712224 12026 raft_consensus.cc:2781] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:29:16.712327 12026 raft_consensus.cc:482] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:29:16.712348 12026 raft_consensus.cc:3037] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 0 FOLLOWER]: Advancing to term 1
W20260812 05:29:16.713100 11715 tablet.cc:2369] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:29:16.713174 12026 raft_consensus.cc:504] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d7e306849c46dda1a853225b0773e3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38549 } } peers { permanent_uuid: "0640c38bd52b469fb522125adca1f371" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39081 } } peers { permanent_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41135 } }
I20260812 05:29:16.713415 12026 leader_election.cc:288] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [CANDIDATE]: Term 1 election: Requested vote from peers 0640c38bd52b469fb522125adca1f371 (127.24.66.67:39081), b5bafe3ecb6e4d758d040b8c9596f1fc (127.24.66.66:41135)
I20260812 05:29:16.713500 11952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "517a37133f214af4a779a429852152f2" candidate_uuid: "60d7e306849c46dda1a853225b0773e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0640c38bd52b469fb522125adca1f371"
I20260812 05:29:16.713589 11952 raft_consensus.cc:3037] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:16.713577 11808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "517a37133f214af4a779a429852152f2" candidate_uuid: "60d7e306849c46dda1a853225b0773e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc"
I20260812 05:29:16.713657 11808 raft_consensus.cc:3037] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:16.714646 11808 raft_consensus.cc:2454] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60d7e306849c46dda1a853225b0773e3 in term 1.
I20260812 05:29:16.714646 11952 raft_consensus.cc:2454] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60d7e306849c46dda1a853225b0773e3 in term 1.
I20260812 05:29:16.714828 11593 leader_election.cc:302] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60d7e306849c46dda1a853225b0773e3, b5bafe3ecb6e4d758d040b8c9596f1fc; no voters:
I20260812 05:29:16.714941 12026 raft_consensus.cc:2781] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:16.715109 12026 raft_consensus.cc:686] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 1 LEADER]: Becoming Leader. State: Replica: 60d7e306849c46dda1a853225b0773e3, State: Running, Role: LEADER
I20260812 05:29:16.715216 12026 consensus_queue.cc:237] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d7e306849c46dda1a853225b0773e3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38549 } } peers { permanent_uuid: "0640c38bd52b469fb522125adca1f371" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39081 } } peers { permanent_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41135 } }
I20260812 05:29:16.716140 11510 catalog_manager.cc:5495] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60d7e306849c46dda1a853225b0773e3 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "60d7e306849c46dda1a853225b0773e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60d7e306849c46dda1a853225b0773e3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0640c38bd52b469fb522125adca1f371" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41135 } health_report { overall_health: UNKNOWN } } }
I20260812 05:29:16.733867 12014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.730768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35318 (local address 127.24.66.65:38549)
0812 05:29:16.730829 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.730831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.730842 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:29:16.731141 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.731143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.731174 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.731276 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:16.731449 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.732012 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.732975 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.733196 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.733293 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.733462 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:16.733569 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:16.733704 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.733738 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.733752 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 05:29:16.735837 11808 raft_consensus.cc:1261] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc [term 1 FOLLOWER]: Refusing update from remote peer 60d7e306849c46dda1a853225b0773e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:16.735875 11952 raft_consensus.cc:1261] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371 [term 1 FOLLOWER]: Refusing update from remote peer 60d7e306849c46dda1a853225b0773e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:16.736245 12029 consensus_queue.cc:1035] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5bafe3ecb6e4d758d040b8c9596f1fc" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 41135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:16.736406 12026 consensus_queue.cc:1035] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0640c38bd52b469fb522125adca1f371" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:16.739136 12035 mvcc.cc:205] Tried to move back new op lower bound from 7317555432387182592 to 7317555432306147328. Current Snapshot: MvccSnapshot[applied={T|T < 7317555432387182592}]
I20260812 05:29:16.739742 12034 mvcc.cc:205] Tried to move back new op lower bound from 7317555432387182592 to 7317555432306147328. Current Snapshot: MvccSnapshot[applied={T|T < 7317555432387182592}]
I20260812 05:29:16.741711 11509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:16.741811 11509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:16.742652 11509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00956b3d9d9f4a48966d275d345eeb2d: Sending DeleteTablet for 3 replicas of tablet 517a37133f214af4a779a429852152f2
I20260812 05:29:16.742986 11643 tablet_service.cc:1514] Processing DeleteTablet for tablet 517a37133f214af4a779a429852152f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:16 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:35316
I20260812 05:29:16.743060 11931 tablet_service.cc:1514] Processing DeleteTablet for tablet 517a37133f214af4a779a429852152f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:16 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:40832
I20260812 05:29:16.743165 11788 tablet_service.cc:1514] Processing DeleteTablet for tablet 517a37133f214af4a779a429852152f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:16 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:53454
I20260812 05:29:16.743278 12043 tablet_replica.cc:330] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3: stopping tablet replica
I20260812 05:29:16.743381 12043 raft_consensus.cc:2229] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:29:16.743489 12043 raft_consensus.cc:2258] T 517a37133f214af4a779a429852152f2 P 60d7e306849c46dda1a853225b0773e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:16.743647 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 11573
I20260812 05:29:16.745286 12045 tablet_replica.cc:330] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: stopping tablet replica
I20260812 05:29:16.745415 12045 raft_consensus.cc:2229] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:29:16.745523 12045 raft_consensus.cc:2258] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:16.743677 12044 tablet_replica.cc:330] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371: stopping tablet replica
I20260812 05:29:16.745723 12044 raft_consensus.cc:2229] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:29:16.745851 12044 raft_consensus.cc:2258] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:16.746639 12044 ts_tablet_manager.cc:1905] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:16.747025 12045 ts_tablet_manager.cc:1905] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:16.748786 12044 ts_tablet_manager.cc:1918] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:16.748795 12045 ts_tablet_manager.cc:1918] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:16.748883 12045 log.cc:1196] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/517a37133f214af4a779a429852152f2
I20260812 05:29:16.748884 12044 log.cc:1196] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/517a37133f214af4a779a429852152f2
I20260812 05:29:16.749189 12045 ts_tablet_manager.cc:1939] T 517a37133f214af4a779a429852152f2 P b5bafe3ecb6e4d758d040b8c9596f1fc: Deleting consensus metadata
I20260812 05:29:16.749189 12044 ts_tablet_manager.cc:1939] T 517a37133f214af4a779a429852152f2 P 0640c38bd52b469fb522125adca1f371: Deleting consensus metadata
I20260812 05:29:16.749665 11494 catalog_manager.cc:4841] TS b5bafe3ecb6e4d758d040b8c9596f1fc (127.24.66.66:41135): tablet 517a37133f214af4a779a429852152f2 (table test-table [id=bb2f0b2c38fb409685e6f105c61e0723]) successfully deleted
I20260812 05:29:16.749761 11494 catalog_manager.cc:4841] TS 0640c38bd52b469fb522125adca1f371 (127.24.66.67:39081): tablet 517a37133f214af4a779a429852152f2 (table test-table [id=bb2f0b2c38fb409685e6f105c61e0723]) successfully deleted
W20260812 05:29:16.753750 11495 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:38549 (error 108)
I20260812 05:29:16.753937 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 11719
I20260812 05:29:16.755316 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.755039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:38549 (local address 127.0.0.1:35320)
0812 05:29:16.755225 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:16.755280 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260812 05:29:16.755529 11495 catalog_manager.cc:4584] TS 60d7e306849c46dda1a853225b0773e3 (127.24.66.65:38549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 517a37133f214af4a779a429852152f2: Network error: Client connection negotiation failed: client connection to 127.24.66.65:38549: connect: Connection refused (error 111)
I20260812 05:29:16.761162 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 11863
I20260812 05:29:16.767570 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 11475
2026-08-12T05:29:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2206 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T05:29:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:16Z Disabled control of system clock
I20260812 05:29:16.801084 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39739
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:39739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:29:16.888177 12052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:16.888432 12052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:16.888455 12052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:16.889622 12052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:16.889662 12052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:16.889675 12052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:16.889688 12052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:16.889705 12052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:16.891067 12052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:39739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:16.891275 12052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:16.891538 12052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:16.896600 12059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:16.896617 12063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:16.896628 12057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:16.896847 12052 server_base.cc:1034] running on GCE node
I20260812 05:29:16.897168 12052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:16.897509 12052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:16.898653 12052 hybrid_clock.cc:648] HybridClock initialized: now 1786512556898617 us; error 58 us; skew 500 ppm
I20260812 05:29:16.899977 12052 webserver.cc:460] Webserver started at http://127.24.66.126:42015/ using document root <none> and password file <none>
I20260812 05:29:16.900292 12052 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:16.900348 12052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:16.900444 12052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:16.901494 12052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "2d66c7fca7a042949d8e2690fd97fab8"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "f0ab5b2512cf3cd02e94c2567e37ed04"
server_key_iv: "6f612e0a880a27349d61ab9a572e1265"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.901799 12052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "2d66c7fca7a042949d8e2690fd97fab8"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "f0ab5b2512cf3cd02e94c2567e37ed04"
server_key_iv: "6f612e0a880a27349d61ab9a572e1265"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.903167 12052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:16.903980 12069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:16.904165 12052 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:16.904234 12052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "2d66c7fca7a042949d8e2690fd97fab8"
format_stamp: "Formatted at 2026-08-12 05:29:16 on dist-test-slave-zpfg"
server_key: "f0ab5b2512cf3cd02e94c2567e37ed04"
server_key_iv: "6f612e0a880a27349d61ab9a572e1265"
server_key_version: "encryptionkey@0"
I20260812 05:29:16.904315 12052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:16.932384 12052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:16.932811 12052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:16.933004 12052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:16.937930 12052 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:39739
I20260812 05:29:16.937999 12122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:39739 every 8 connection(s)
I20260812 05:29:16.938396 12052 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:16.939620 12123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:16.942171 12123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8: Bootstrap starting.
I20260812 05:29:16.942885 12123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:16.943166 12123 log.cc:824] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:16.944123 12123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8: No bootstrap required, opened a new log
I20260812 05:29:16.945967 12123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d66c7fca7a042949d8e2690fd97fab8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39739 } }
I20260812 05:29:16.946132 12123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:16.946164 12123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d66c7fca7a042949d8e2690fd97fab8, State: Initialized, Role: FOLLOWER
I20260812 05:29:16.946322 12123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d66c7fca7a042949d8e2690fd97fab8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39739 } }
I20260812 05:29:16.946470 12123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:16.946497 12123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:16.946532 12123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:16.946938 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 12052
I20260812 05:29:16.947029 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:16.947218 24841 external_mini_cluster.cc:1428] Setting key da81710f38e516fa04bee87c541dc72e
I20260812 05:29:16.947212 12123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d66c7fca7a042949d8e2690fd97fab8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39739 } }
I20260812 05:29:16.947350 12123 leader_election.cc:302] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d66c7fca7a042949d8e2690fd97fab8; no voters:
I20260812 05:29:16.947669 12123 leader_election.cc:288] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:16.947710 12126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:16.948010 12126 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [term 1 LEADER]: Becoming Leader. State: Replica: 2d66c7fca7a042949d8e2690fd97fab8, State: Running, Role: LEADER
I20260812 05:29:16.948045 12123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:16.948189 12126 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d66c7fca7a042949d8e2690fd97fab8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39739 } }
I20260812 05:29:16.949398 12127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d66c7fca7a042949d8e2690fd97fab8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d66c7fca7a042949d8e2690fd97fab8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39739 } } }
I20260812 05:29:16.949498 12127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:16.949730 12140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:16.949767 12129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d66c7fca7a042949d8e2690fd97fab8. Latest consensus state: current_term: 1 leader_uuid: "2d66c7fca7a042949d8e2690fd97fab8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d66c7fca7a042949d8e2690fd97fab8" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39739 } } }
I20260812 05:29:16.949841 12129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:16.950539 12140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:16.952298 12140 catalog_manager.cc:1347] Generated new cluster ID: 04502bf9d7f24d95b18dde5fff37a5f9
I20260812 05:29:16.952359 12140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:16.954217 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:16.947876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59444 (local address 127.24.66.126:39739)
0812 05:29:16.948136 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:16.948148 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:16.948190 (+ 42us) server_negotiation.cc:407] Connection header received
0812 05:29:16.948460 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:16.948466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:16.948544 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:16.948697 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:16.949045 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.950715 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.951869 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:16.952521 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:16.952677 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:16.953490 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:16.953494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:16.953494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:16.953699 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:16.953737 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:16.953749 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:16.953840 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:16.953922 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 05:29:16.954013 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":111,"threads_started":1}
I20260812 05:29:16.973446 12140 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:16.974293 12140 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:16.983230 12140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8: Generated new TSK 0
I20260812 05:29:16.983523 12140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:16.987738 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39739
--builtin_ntp_servers=127.24.66.84:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:29:17.095340 12148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:17.095557 12148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:17.095582 12148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:17.096858 12148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:17.096915 12148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:17.096942 12148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:29:17.098464 12148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:17.098738 12148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:17.099078 12148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:17.104812 12154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:17.104851 12156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:17.104812 12153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:17.105479 12148 server_base.cc:1034] running on GCE node
I20260812 05:29:17.105753 12148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:17.106026 12148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:17.107187 12148 hybrid_clock.cc:648] HybridClock initialized: now 1786512557107172 us; error 34 us; skew 500 ppm
I20260812 05:29:17.108774 12148 webserver.cc:460] Webserver started at http://127.24.66.65:39007/ using document root <none> and password file <none>
I20260812 05:29:17.109066 12148 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:17.109118 12148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:17.109269 12148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:17.110466 12148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "1176f1c270a443d3936a638b24fea8c3"
format_stamp: "Formatted at 2026-08-12 05:29:17 on dist-test-slave-zpfg"
server_key: "8a9955c647c104940b381823b6ccf84c"
server_key_iv: "e9ea095ad58eb74c29db77a94bd6ca37"
server_key_version: "encryptionkey@0"
I20260812 05:29:17.110860 12148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "1176f1c270a443d3936a638b24fea8c3"
format_stamp: "Formatted at 2026-08-12 05:29:17 on dist-test-slave-zpfg"
server_key: "8a9955c647c104940b381823b6ccf84c"
server_key_iv: "e9ea095ad58eb74c29db77a94bd6ca37"
server_key_version: "encryptionkey@0"
I20260812 05:29:17.112386 12148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:29:17.113370 12162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:17.113654 12148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:17.113799 12148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "1176f1c270a443d3936a638b24fea8c3"
format_stamp: "Formatted at 2026-08-12 05:29:17 on dist-test-slave-zpfg"
server_key: "8a9955c647c104940b381823b6ccf84c"
server_key_iv: "e9ea095ad58eb74c29db77a94bd6ca37"
server_key_version: "encryptionkey@0"
I20260812 05:29:17.113890 12148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:17.131402 12148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:17.131805 12148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:17.131991 12148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:17.132258 12148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:17.132632 12148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:17.132690 12148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:17.132735 12148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:17.132759 12148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:17.139254 12148 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:38749
I20260812 05:29:17.139353 12282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:38749 every 8 connection(s)
I20260812 05:29:17.139725 12148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:29:17.145051 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 12148
I20260812 05:29:17.145193 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:29:17.145434 24841 external_mini_cluster.cc:1428] Setting key a0b37fec6deb2ebe211232099ce6d266
I20260812 05:29:17.145949 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.140640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:48515 (local address 127.24.66.126:39739)
0812 05:29:17.140722 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:17.140724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:17.140943 (+ 219us) server_negotiation.cc:407] Connection header received
0812 05:29:17.141469 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:17.141471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:17.141484 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:17.141557 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:17.142324 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.142902 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.143967 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.144177 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.144237 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.145563 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:17.145565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:17.145565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:17.145572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:17.145590 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:17.145597 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:17.145666 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:17.145813 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 05:29:17.145853 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 05:29:17.146185 12286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.140588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39739 (local address 127.24.66.65:48515)
0812 05:29:17.140803 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:17.140847 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:17.141351 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:17.141641 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:17.141645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:17.141654 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:17.142204 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.142209 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.142998 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.142999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.143875 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.143878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.144275 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.144276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.144402 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.145373 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:29:17.145373 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:17.145407 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:29:17.145432 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:17.145692 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:17.145700 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:17.145704 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:29:17.145805 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 05:29:17.145940 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260812 05:29:17.146708 12283 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39739
I20260812 05:29:17.146746 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39739
--builtin_ntp_servers=127.24.66.84:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 05:29:17.146831 12283 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:17.147073 12283 heartbeater.cc:511] Master 127.24.66.126:39739 requested a full tablet report, sending...
I20260812 05:29:17.147696 12086 ts_manager.cc:194] Registered new tserver with Master: 1176f1c270a443d3936a638b24fea8c3 (127.24.66.65:38749)
I20260812 05:29:17.148669 12086 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.65:48515
W20260812 05:29:17.233749 12287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:17.233979 12287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:17.234002 12287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:17.235242 12287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:17.235298 12287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:17.235325 12287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:29:17.236671 12287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:17.236877 12287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:17.237149 12287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:17.242704 12294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:17.242869 12287 server_base.cc:1034] running on GCE node
W20260812 05:29:17.242735 12296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:17.242729 12293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:17.243240 12287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:17.243531 12287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:17.244722 12287 hybrid_clock.cc:648] HybridClock initialized: now 1786512557244711 us; error 35 us; skew 500 ppm
I20260812 05:29:17.246181 12287 webserver.cc:460] Webserver started at http://127.24.66.66:36489/ using document root <none> and password file <none>
I20260812 05:29:17.246549 12287 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:17.246628 12287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:17.246773 12287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:17.248010 12287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "25998228e79747cdabf25ddac96f9cf2"
format_stamp: "Formatted at 2026-08-12 05:29:17 on dist-test-slave-zpfg"
server_key: "0028b853102483794da5a2df1d7b7aaf"
server_key_iv: "846fac416589d5f3c8227b824b4a1d30"
server_key_version: "encryptionkey@0"
I20260812 05:29:17.248416 12287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "25998228e79747cdabf25ddac96f9cf2"
format_stamp: "Formatted at 2026-08-12 05:29:17 on dist-test-slave-zpfg"
server_key: "0028b853102483794da5a2df1d7b7aaf"
server_key_iv: "846fac416589d5f3c8227b824b4a1d30"
server_key_version: "encryptionkey@0"
I20260812 05:29:17.250061 12287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:17.250998 12302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:17.251216 12287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:29:17.251327 12287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "25998228e79747cdabf25ddac96f9cf2"
format_stamp: "Formatted at 2026-08-12 05:29:17 on dist-test-slave-zpfg"
server_key: "0028b853102483794da5a2df1d7b7aaf"
server_key_iv: "846fac416589d5f3c8227b824b4a1d30"
server_key_version: "encryptionkey@0"
I20260812 05:29:17.251425 12287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:17.265609 12287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:17.265944 12287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:17.266062 12287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:17.266258 12287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:17.266666 12287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:17.266731 12287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:17.266793 12287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:17.266872 12287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:17.273856 12287 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:34553
I20260812 05:29:17.273967 12425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:34553 every 8 connection(s)
I20260812 05:29:17.274282 12287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:29:17.279933 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.275154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:54773 (local address 127.24.66.126:39739)
0812 05:29:17.275231 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:17.275233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:17.275514 (+ 281us) server_negotiation.cc:407] Connection header received
0812 05:29:17.276025 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:17.276026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:17.276037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:17.276126 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:17.276814 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.277491 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.278545 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.278757 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.278824 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.279671 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:17.279672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:17.279672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:17.279678 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:17.279695 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:17.279699 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:17.279749 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:17.279832 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 05:29:17.279860 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:29:17.280154 12428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.275123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39739 (local address 127.24.66.66:54773)
0812 05:29:17.275324 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:17.275434 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:17.275932 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:17.276185 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:17.276189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:17.276199 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:17.276727 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.276732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.277583 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.277584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.278461 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.278464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.278841 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.278842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.278951 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.279527 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:29:17.279528 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:17.279558 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:29:17.279582 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:17.279759 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:17.279763 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:17.279765 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:29:17.279819 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 05:29:17.279902 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260812 05:29:17.280844 12426 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39739
I20260812 05:29:17.280948 12426 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:17.281112 12426 heartbeater.cc:511] Master 127.24.66.126:39739 requested a full tablet report, sending...
I20260812 05:29:17.281504 12086 ts_manager.cc:194] Registered new tserver with Master: 25998228e79747cdabf25ddac96f9cf2 (127.24.66.66:34553)
I20260812 05:29:17.281862 12086 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:54773
I20260812 05:29:17.283489 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 12287
I20260812 05:29:17.283644 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:29:17.283939 24841 external_mini_cluster.cc:1428] Setting key 2a0292793a0ea953678f88f537515085
I20260812 05:29:17.285181 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39739
--builtin_ntp_servers=127.24.66.84:39051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:29:17.370041 12429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:17.370273 12429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:17.370296 12429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:17.371567 12429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:17.371619 12429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:17.371646 12429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:29:17.372989 12429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:39051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:39739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:17.373207 12429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:17.373497 12429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:17.379141 12435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:17.379197 12429 server_base.cc:1034] running on GCE node
W20260812 05:29:17.379127 12436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:17.379127 12438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:17.379716 12429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:17.379992 12429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:17.381165 12429 hybrid_clock.cc:648] HybridClock initialized: now 1786512557381152 us; error 48 us; skew 500 ppm
I20260812 05:29:17.382586 12429 webserver.cc:460] Webserver started at http://127.24.66.67:38199/ using document root <none> and password file <none>
I20260812 05:29:17.382862 12429 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:17.382938 12429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:17.383096 12429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:17.384285 12429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "f38128a16054455d9f63d60df64c219b"
format_stamp: "Formatted at 2026-08-12 05:29:17 on dist-test-slave-zpfg"
server_key: "4ce4d0a363afa924cb0c53ee9a33ea4c"
server_key_iv: "891665d27c75875a3469afa276055195"
server_key_version: "encryptionkey@0"
I20260812 05:29:17.384706 12429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "f38128a16054455d9f63d60df64c219b"
format_stamp: "Formatted at 2026-08-12 05:29:17 on dist-test-slave-zpfg"
server_key: "4ce4d0a363afa924cb0c53ee9a33ea4c"
server_key_iv: "891665d27c75875a3469afa276055195"
server_key_version: "encryptionkey@0"
I20260812 05:29:17.386193 12429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:17.387189 12444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:17.387383 12429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:17.387496 12429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "f38128a16054455d9f63d60df64c219b"
format_stamp: "Formatted at 2026-08-12 05:29:17 on dist-test-slave-zpfg"
server_key: "4ce4d0a363afa924cb0c53ee9a33ea4c"
server_key_iv: "891665d27c75875a3469afa276055195"
server_key_version: "encryptionkey@0"
I20260812 05:29:17.387579 12429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:17.401752 12429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:17.402128 12429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:17.402287 12429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:17.402626 12429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:17.403010 12429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:17.403070 12429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:17.403106 12429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:17.403152 12429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:17.409910 12429 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:42471
I20260812 05:29:17.409962 12562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:42471 every 8 connection(s)
I20260812 05:29:17.410399 12429 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:29:17.411192 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 12429
I20260812 05:29:17.411286 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:29:17.411551 24841 external_mini_cluster.cc:1428] Setting key 66cefa894985830ee12679c4b019c066
I20260812 05:29:17.416990 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.411390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:43773 (local address 127.24.66.126:39739)
0812 05:29:17.411683 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:17.411685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:17.411760 (+ 75us) server_negotiation.cc:407] Connection header received
0812 05:29:17.412711 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:17.412713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:17.412727 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:17.412786 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:17.413577 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.414181 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.415432 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.415675 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.415759 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.416690 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:17.416692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:17.416693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:17.416699 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:17.416717 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:17.416722 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:17.416784 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:17.416866 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 05:29:17.416902 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260812 05:29:17.417348 12566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.411379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:39739 (local address 127.24.66.67:43773)
0812 05:29:17.411623 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:17.411674 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:17.412525 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:17.412869 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:17.412874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:17.412893 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:17.413462 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.413467 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.414290 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.414291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.415331 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.415333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.415783 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.415784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.415947 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.416500 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:29:17.416500 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:17.416531 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:29:17.416555 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:17.416787 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:17.416794 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:17.416798 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:29:17.416872 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 05:29:17.416982 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":117,"threads_started":1}
I20260812 05:29:17.417896 12563 heartbeater.cc:347] Connected to a master server at 127.24.66.126:39739
I20260812 05:29:17.417999 12563 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:17.418175 12563 heartbeater.cc:511] Master 127.24.66.126:39739 requested a full tablet report, sending...
I20260812 05:29:17.418605 12086 ts_manager.cc:194] Registered new tserver with Master: f38128a16054455d9f63d60df64c219b (127.24.66.67:42471)
I20260812 05:29:17.419003 12086 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:43773
I20260812 05:29:17.423403 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:29:17.428007 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.424891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44086 (local address 127.24.66.126:39739)
0812 05:29:17.424957 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:17.424962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:17.424988 (+ 26us) server_negotiation.cc:407] Connection header received
0812 05:29:17.425030 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:17.425031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:17.425041 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:17.425104 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:17.425293 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.425855 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.426723 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.426927 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.426992 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.427723 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:17.427725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:17.427725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:17.427729 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:17.427743 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:17.427747 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:17.427848 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:17.427937 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 05:29:17.427948 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:29:17.429880 12086 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:29:17.431699 12086 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:29:17.439239 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.434559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37694 (local address 127.24.66.65:38749)
0812 05:29:17.434836 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:17.434840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:17.434993 (+ 153us) server_negotiation.cc:407] Connection header received
0812 05:29:17.435045 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:17.435048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:17.435071 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:17.435166 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:17.435367 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.436567 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.438036 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.438768 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.438861 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.438966 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:17.439096 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 05:29:17.439142 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":97,"threads_started":1}
I20260812 05:29:17.439244 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.434473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:38749 (local address 127.0.0.1:37694)
0812 05:29:17.434821 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:17.434860 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:17.434925 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:17.435174 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:17.435177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:17.435179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:17.435269 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.435271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.436675 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.436676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.437936 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.437939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.438878 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.438879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.438966 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.438982 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 05:29:17.439071 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 05:29:17.439137 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":140,"thread_start_us":68,"threads_started":1}
I20260812 05:29:17.439745 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.434700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:34553 (local address 127.0.0.1:35544)
0812 05:29:17.434919 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:17.434928 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:17.434972 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:17.435471 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:17.435472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:17.435475 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:17.435541 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.435543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.436651 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.436653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.438246 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.438248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.439488 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.439490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.439560 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.439565 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:29:17.439620 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 05:29:17.439669 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":80,"thread_start_us":73,"threads_started":1}
I20260812 05:29:17.439985 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.434915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35544 (local address 127.24.66.66:34553)
0812 05:29:17.435113 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:17.435118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:17.435269 (+ 151us) server_negotiation.cc:407] Connection header received
0812 05:29:17.435311 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:17.435313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:17.435330 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:17.435405 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:17.435625 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.436418 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.438422 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.439123 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.439327 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.439444 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:17.439852 (+ 408us) server_negotiation.cc:299] Negotiation successful
0812 05:29:17.439898 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":87,"threads_started":1}
I20260812 05:29:17.440058 12572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.434463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:42471 (local address 127.0.0.1:39658)
0812 05:29:17.434765 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:17.434814 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:17.434898 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:17.435385 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:17.435386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:17.435388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:17.435429 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.435430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.436739 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.436740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.438229 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.438232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.439621 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.439622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.439694 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.439700 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 05:29:17.439733 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 05:29:17.439776 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":185,"thread_start_us":82,"threads_started":1}
I20260812 05:29:17.440151 12213 tablet_service.cc:1467] Processing CreateTablet for tablet 88316e5acf7645c48799663a4e9810f4 (DEFAULT_TABLE table=test-table [id=bf6fd1d4e7ae430bb13fe32d7973ab4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:17.440460 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.434573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39658 (local address 127.24.66.67:42471)
0812 05:29:17.434739 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:17.434745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:17.435074 (+ 329us) server_negotiation.cc:407] Connection header received
0812 05:29:17.435125 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:17.435128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:17.435145 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:17.435266 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:17.435513 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.436618 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.438330 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.439515 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.440172 (+ 657us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.440278 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:17.440304 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 05:29:17.440359 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260812 05:29:17.440608 12213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88316e5acf7645c48799663a4e9810f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:17.440676 12352 tablet_service.cc:1467] Processing CreateTablet for tablet 88316e5acf7645c48799663a4e9810f4 (DEFAULT_TABLE table=test-table [id=bf6fd1d4e7ae430bb13fe32d7973ab4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:17.441030 12352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88316e5acf7645c48799663a4e9810f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:17.441118 12495 tablet_service.cc:1467] Processing CreateTablet for tablet 88316e5acf7645c48799663a4e9810f4 (DEFAULT_TABLE table=test-table [id=bf6fd1d4e7ae430bb13fe32d7973ab4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:17.441473 12495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88316e5acf7645c48799663a4e9810f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:17.444262 12580 tablet_bootstrap.cc:492] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b: Bootstrap starting.
I20260812 05:29:17.444967 12581 tablet_bootstrap.cc:492] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3: Bootstrap starting.
I20260812 05:29:17.445086 12580 tablet_bootstrap.cc:654] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:17.445403 12580 log.cc:824] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:17.445732 12581 tablet_bootstrap.cc:654] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:17.446097 12581 log.cc:824] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:17.446233 12580 tablet_bootstrap.cc:492] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b: No bootstrap required, opened a new log
I20260812 05:29:17.446322 12580 ts_tablet_manager.cc:1397] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:29:17.446523 12582 tablet_bootstrap.cc:492] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: Bootstrap starting.
I20260812 05:29:17.446978 12581 tablet_bootstrap.cc:492] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3: No bootstrap required, opened a new log
I20260812 05:29:17.447069 12581 ts_tablet_manager.cc:1397] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 05:29:17.447373 12582 tablet_bootstrap.cc:654] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:17.447773 12582 log.cc:824] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:17.448784 12582 tablet_bootstrap.cc:492] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: No bootstrap required, opened a new log
I20260812 05:29:17.448904 12582 ts_tablet_manager.cc:1397] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:29:17.449177 12580 raft_consensus.cc:348] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38128a16054455d9f63d60df64c219b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 42471 } } peers { permanent_uuid: "1176f1c270a443d3936a638b24fea8c3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38749 } } peers { permanent_uuid: "25998228e79747cdabf25ddac96f9cf2" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34553 } }
I20260812 05:29:17.449359 12580 raft_consensus.cc:374] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:17.449412 12580 raft_consensus.cc:729] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f38128a16054455d9f63d60df64c219b, State: Initialized, Role: FOLLOWER
I20260812 05:29:17.449575 12580 consensus_queue.cc:260] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38128a16054455d9f63d60df64c219b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 42471 } } peers { permanent_uuid: "1176f1c270a443d3936a638b24fea8c3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38749 } } peers { permanent_uuid: "25998228e79747cdabf25ddac96f9cf2" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34553 } }
I20260812 05:29:17.449996 12580 ts_tablet_manager.cc:1428] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 05:29:17.450034 12563 heartbeater.cc:503] Master 127.24.66.126:39739 was elected leader, sending a full tablet report...
I20260812 05:29:17.450294 12581 raft_consensus.cc:348] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38128a16054455d9f63d60df64c219b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 42471 } } peers { permanent_uuid: "1176f1c270a443d3936a638b24fea8c3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38749 } } peers { permanent_uuid: "25998228e79747cdabf25ddac96f9cf2" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34553 } }
I20260812 05:29:17.450560 12581 raft_consensus.cc:374] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:17.450611 12581 raft_consensus.cc:729] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1176f1c270a443d3936a638b24fea8c3, State: Initialized, Role: FOLLOWER
I20260812 05:29:17.450727 12581 consensus_queue.cc:260] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38128a16054455d9f63d60df64c219b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 42471 } } peers { permanent_uuid: "1176f1c270a443d3936a638b24fea8c3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38749 } } peers { permanent_uuid: "25998228e79747cdabf25ddac96f9cf2" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34553 } }
I20260812 05:29:17.451013 12581 ts_tablet_manager.cc:1428] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 05:29:17.451200 12283 heartbeater.cc:503] Master 127.24.66.126:39739 was elected leader, sending a full tablet report...
I20260812 05:29:17.452093 12582 raft_consensus.cc:348] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38128a16054455d9f63d60df64c219b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 42471 } } peers { permanent_uuid: "1176f1c270a443d3936a638b24fea8c3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38749 } } peers { permanent_uuid: "25998228e79747cdabf25ddac96f9cf2" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34553 } }
I20260812 05:29:17.452227 12582 raft_consensus.cc:374] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:17.452250 12582 raft_consensus.cc:729] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25998228e79747cdabf25ddac96f9cf2, State: Initialized, Role: FOLLOWER
I20260812 05:29:17.452368 12582 consensus_queue.cc:260] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38128a16054455d9f63d60df64c219b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 42471 } } peers { permanent_uuid: "1176f1c270a443d3936a638b24fea8c3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38749 } } peers { permanent_uuid: "25998228e79747cdabf25ddac96f9cf2" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34553 } }
I20260812 05:29:17.452639 12582 ts_tablet_manager.cc:1428] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260812 05:29:17.452662 12426 heartbeater.cc:503] Master 127.24.66.126:39739 was elected leader, sending a full tablet report...
W20260812 05:29:17.524940 12427 tablet.cc:2369] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:29:17.532426 12587 raft_consensus.cc:482] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:29:17.532558 12587 raft_consensus.cc:504] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38128a16054455d9f63d60df64c219b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 42471 } } peers { permanent_uuid: "1176f1c270a443d3936a638b24fea8c3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38749 } } peers { permanent_uuid: "25998228e79747cdabf25ddac96f9cf2" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34553 } }
I20260812 05:29:17.532971 12587 leader_election.cc:288] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f38128a16054455d9f63d60df64c219b (127.24.66.67:42471), 25998228e79747cdabf25ddac96f9cf2 (127.24.66.66:34553)
I20260812 05:29:17.536589 12286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.533312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:42471 (local address 127.24.66.65:36287)
0812 05:29:17.533383 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:17.533416 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:17.533482 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:17.533672 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:17.533673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:17.533676 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:17.533755 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.533758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.534561 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.534562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.535565 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.535569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.536215 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.536216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.536380 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.536385 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:29:17.536457 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 05:29:17.536510 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 05:29:17.536607 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.533460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:36287 (local address 127.24.66.67:42471)
0812 05:29:17.533515 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:17.533517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:17.533525 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:29:17.533558 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:17.533558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:17.533569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:17.533669 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:17.533883 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.534481 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.535666 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.536123 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.536191 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.536200 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:17.536480 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 05:29:17.536526 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 05:29:17.536770 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.533494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:34553 (local address 127.24.66.65:42235)
0812 05:29:17.533695 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:17.533705 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:17.533784 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:17.533974 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:17.533975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:17.533976 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:17.534048 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.534049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.534970 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.534971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.535796 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:17.535798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.536488 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.536489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:17.536571 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.536576 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:29:17.536638 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 05:29:17.536678 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":123,"threads_started":1}
I20260812 05:29:17.536810 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.533550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:42235 (local address 127.24.66.66:34553)
0812 05:29:17.533606 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:17.533609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:17.533772 (+ 163us) server_negotiation.cc:407] Connection header received
0812 05:29:17.533878 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:17.533880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:17.533894 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:17.533957 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:17.534126 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.534872 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.535886 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.536384 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.536468 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.536475 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:17.536660 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 05:29:17.536709 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 05:29:17.537029 12516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88316e5acf7645c48799663a4e9810f4" candidate_uuid: "1176f1c270a443d3936a638b24fea8c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f38128a16054455d9f63d60df64c219b" is_pre_election: true
I20260812 05:29:17.537177 12516 raft_consensus.cc:2454] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1176f1c270a443d3936a638b24fea8c3 in term 0.
I20260812 05:29:17.537487 12165 leader_election.cc:302] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1176f1c270a443d3936a638b24fea8c3, f38128a16054455d9f63d60df64c219b; no voters:
I20260812 05:29:17.537623 12587 raft_consensus.cc:2781] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:29:17.537695 12587 raft_consensus.cc:482] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:29:17.537716 12587 raft_consensus.cc:3037] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:17.537626 12373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88316e5acf7645c48799663a4e9810f4" candidate_uuid: "1176f1c270a443d3936a638b24fea8c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25998228e79747cdabf25ddac96f9cf2" is_pre_election: true
I20260812 05:29:17.537811 12373 raft_consensus.cc:2454] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1176f1c270a443d3936a638b24fea8c3 in term 0.
I20260812 05:29:17.538709 12587 raft_consensus.cc:504] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38128a16054455d9f63d60df64c219b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 42471 } } peers { permanent_uuid: "1176f1c270a443d3936a638b24fea8c3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38749 } } peers { permanent_uuid: "25998228e79747cdabf25ddac96f9cf2" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34553 } }
I20260812 05:29:17.538872 12587 leader_election.cc:288] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [CANDIDATE]: Term 1 election: Requested vote from peers f38128a16054455d9f63d60df64c219b (127.24.66.67:42471), 25998228e79747cdabf25ddac96f9cf2 (127.24.66.66:34553)
I20260812 05:29:17.539099 12373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88316e5acf7645c48799663a4e9810f4" candidate_uuid: "1176f1c270a443d3936a638b24fea8c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25998228e79747cdabf25ddac96f9cf2"
I20260812 05:29:17.539206 12373 raft_consensus.cc:3037] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:17.539111 12516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88316e5acf7645c48799663a4e9810f4" candidate_uuid: "1176f1c270a443d3936a638b24fea8c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f38128a16054455d9f63d60df64c219b"
I20260812 05:29:17.539316 12516 raft_consensus.cc:3037] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:17.540354 12373 raft_consensus.cc:2454] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1176f1c270a443d3936a638b24fea8c3 in term 1.
I20260812 05:29:17.540354 12516 raft_consensus.cc:2454] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1176f1c270a443d3936a638b24fea8c3 in term 1.
I20260812 05:29:17.540587 12165 leader_election.cc:302] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1176f1c270a443d3936a638b24fea8c3, 25998228e79747cdabf25ddac96f9cf2; no voters:
I20260812 05:29:17.540704 12587 raft_consensus.cc:2781] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:17.540881 12587 raft_consensus.cc:686] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [term 1 LEADER]: Becoming Leader. State: Replica: 1176f1c270a443d3936a638b24fea8c3, State: Running, Role: LEADER
I20260812 05:29:17.540995 12587 consensus_queue.cc:237] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38128a16054455d9f63d60df64c219b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 42471 } } peers { permanent_uuid: "1176f1c270a443d3936a638b24fea8c3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38749 } } peers { permanent_uuid: "25998228e79747cdabf25ddac96f9cf2" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34553 } }
I20260812 05:29:17.542003 12086 catalog_manager.cc:5495] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1176f1c270a443d3936a638b24fea8c3 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "1176f1c270a443d3936a638b24fea8c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f38128a16054455d9f63d60df64c219b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 42471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1176f1c270a443d3936a638b24fea8c3" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 38749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25998228e79747cdabf25ddac96f9cf2" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34553 } health_report { overall_health: UNKNOWN } } }
I20260812 05:29:17.559620 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.556986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37710 (local address 127.24.66.65:38749)
0812 05:29:17.557068 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:17.557070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:17.557080 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:29:17.557119 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:17.557120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:17.557211 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:17.557319 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:17.557485 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.557966 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.558818 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:17.559014 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:17.559101 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:17.559268 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:17.559358 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:17.559470 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:17.559500 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 05:29:17.559511 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 05:29:17.561692 12373 raft_consensus.cc:1261] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2 [term 1 FOLLOWER]: Refusing update from remote peer 1176f1c270a443d3936a638b24fea8c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:17.561735 12516 raft_consensus.cc:1261] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b [term 1 FOLLOWER]: Refusing update from remote peer 1176f1c270a443d3936a638b24fea8c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:17.562076 12587 consensus_queue.cc:1035] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25998228e79747cdabf25ddac96f9cf2" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:17.562237 12590 consensus_queue.cc:1035] T 88316e5acf7645c48799663a4e9810f4 P 1176f1c270a443d3936a638b24fea8c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f38128a16054455d9f63d60df64c219b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:17.565104 12597 mvcc.cc:205] Tried to move back new op lower bound from 7317555435769851904 to 7317555435688738816. Current Snapshot: MvccSnapshot[applied={T|T < 7317555435769851904}]
I20260812 05:29:17.565310 12594 mvcc.cc:205] Tried to move back new op lower bound from 7317555435769851904 to 7317555435688738816. Current Snapshot: MvccSnapshot[applied={T|T < 7317555435769851904}]
I20260812 05:29:17.568326 12085 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:17.568436 12085 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:17.569330 12085 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d66c7fca7a042949d8e2690fd97fab8: Sending DeleteTablet for 3 replicas of tablet 88316e5acf7645c48799663a4e9810f4
I20260812 05:29:17.569684 12213 tablet_service.cc:1514] Processing DeleteTablet for tablet 88316e5acf7645c48799663a4e9810f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:17 UTC) from {username='slave'} at 127.0.0.1:37694
I20260812 05:29:17.569775 12495 tablet_service.cc:1514] Processing DeleteTablet for tablet 88316e5acf7645c48799663a4e9810f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:17 UTC) from {username='slave'} at 127.0.0.1:39658
I20260812 05:29:17.569998 12352 tablet_service.cc:1514] Processing DeleteTablet for tablet 88316e5acf7645c48799663a4e9810f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:17 UTC) from {username='slave'} at 127.0.0.1:35544
I20260812 05:29:17.570031 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 12148
I20260812 05:29:17.575672 12606 tablet_replica.cc:330] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: stopping tablet replica
I20260812 05:29:17.575824 12605 tablet_replica.cc:330] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b: stopping tablet replica
I20260812 05:29:17.575838 12606 raft_consensus.cc:2229] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:29:17.575976 12605 raft_consensus.cc:2229] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:29:17.575989 12606 raft_consensus.cc:2258] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:17.576110 12605 raft_consensus.cc:2258] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:17.576581 12606 ts_tablet_manager.cc:1905] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:17.576601 12605 ts_tablet_manager.cc:1905] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:17.578482 12605 ts_tablet_manager.cc:1918] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:17.578586 12605 log.cc:1196] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/88316e5acf7645c48799663a4e9810f4
I20260812 05:29:17.578735 12606 ts_tablet_manager.cc:1918] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:17.578826 12606 log.cc:1196] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/88316e5acf7645c48799663a4e9810f4
I20260812 05:29:17.578910 12605 ts_tablet_manager.cc:1939] T 88316e5acf7645c48799663a4e9810f4 P f38128a16054455d9f63d60df64c219b: Deleting consensus metadata
I20260812 05:29:17.579083 12606 ts_tablet_manager.cc:1939] T 88316e5acf7645c48799663a4e9810f4 P 25998228e79747cdabf25ddac96f9cf2: Deleting consensus metadata
I20260812 05:29:17.579444 12072 catalog_manager.cc:4841] TS f38128a16054455d9f63d60df64c219b (127.24.66.67:42471): tablet 88316e5acf7645c48799663a4e9810f4 (table test-table [id=bf6fd1d4e7ae430bb13fe32d7973ab4c]) successfully deleted
I20260812 05:29:17.579586 12072 catalog_manager.cc:4841] TS 25998228e79747cdabf25ddac96f9cf2 (127.24.66.66:34553): tablet 88316e5acf7645c48799663a4e9810f4 (table test-table [id=bf6fd1d4e7ae430bb13fe32d7973ab4c]) successfully deleted
W20260812 05:29:17.580535 12070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:38749 (error 108)
I20260812 05:29:17.581059 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:17.580894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:38749 (local address 127.0.0.1:37712)
0812 05:29:17.580970 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:17.581017 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:38749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 05:29:17.581132 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 12287
W20260812 05:29:17.581199 12070 catalog_manager.cc:4584] TS 1176f1c270a443d3936a638b24fea8c3 (127.24.66.65:38749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88316e5acf7645c48799663a4e9810f4: Network error: Client connection negotiation failed: client connection to 127.24.66.65:38749: connect: Connection refused (error 111)
I20260812 05:29:17.587795 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 12429
I20260812 05:29:17.593565 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 12052
2026-08-12T05:29:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (824 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:17 dist-test-slave-zpfg krb5kdc[12610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:17 dist-test-slave-zpfg krb5kdc[12610](info): set up 2 sockets
Aug 12 05:29:17 dist-test-slave-zpfg krb5kdc[12610](info): commencing operation
krb5kdc: starting...
W20260812 05:29:18.707499 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:18 dist-test-slave-zpfg krb5kdc[12610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:29:18.845824 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:44909
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:44909
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:18.937930 12624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:18.938164 12624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:18.938189 12624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:18.939465 12624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:18.939514 12624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:18.939529 12624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:18.939543 12624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:18.939555 12624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:18.941108 12624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:44909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:44909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:18.941409 12624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:18.941665 12624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:18.946920 12630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:18.946940 12629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:18.947077 12632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:18.947345 12624 server_base.cc:1034] running on GCE node
I20260812 05:29:18.947607 12624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:18.947908 12624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:18.949059 12624 hybrid_clock.cc:648] HybridClock initialized: now 1786512558949030 us; error 43 us; skew 500 ppm
Aug 12 05:29:18 dist-test-slave-zpfg krb5kdc[12610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512558, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:18.957286 12624 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:29:18.957842 12624 webserver.cc:460] Webserver started at http://127.24.66.126:39793/ using document root <none> and password file <none>
I20260812 05:29:18.958101 12624 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:18.958184 12624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:18.958388 12624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:18.959621 12624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "6386869496b244cf9786f4f0998e0d1c"
format_stamp: "Formatted at 2026-08-12 05:29:18 on dist-test-slave-zpfg"
server_key: "cb32ebba51580d6233556cb4fff0936c"
server_key_iv: "a362530f1ccf95343625b4b50debc0ee"
server_key_version: "encryptionkey@0"
I20260812 05:29:18.960028 12624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "6386869496b244cf9786f4f0998e0d1c"
format_stamp: "Formatted at 2026-08-12 05:29:18 on dist-test-slave-zpfg"
server_key: "cb32ebba51580d6233556cb4fff0936c"
server_key_iv: "a362530f1ccf95343625b4b50debc0ee"
server_key_version: "encryptionkey@0"
I20260812 05:29:18.961555 12624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:29:18.962622 12639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:18.962921 12624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:18.963138 12624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "6386869496b244cf9786f4f0998e0d1c"
format_stamp: "Formatted at 2026-08-12 05:29:18 on dist-test-slave-zpfg"
server_key: "cb32ebba51580d6233556cb4fff0936c"
server_key_iv: "a362530f1ccf95343625b4b50debc0ee"
server_key_version: "encryptionkey@0"
I20260812 05:29:18.963222 12624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:18.973377 12624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:18.976061 12624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:18.976271 12624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:18.981439 12624 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:44909
I20260812 05:29:18.981477 12694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:44909 every 8 connection(s)
I20260812 05:29:18.981885 12624 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:18.982600 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 12624
I20260812 05:29:18.982784 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:18.983023 24841 external_mini_cluster.cc:1428] Setting key e118c1907b722748197f469ed5dab946
I20260812 05:29:18.983260 12695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:18.986970 12695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c: Bootstrap starting.
I20260812 05:29:18.987748 12695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:18.988086 12695 log.cc:824] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:18.988858 12695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c: No bootstrap required, opened a new log
I20260812 05:29:18.991117 12695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6386869496b244cf9786f4f0998e0d1c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44909 } }
I20260812 05:29:18.991274 12695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:18.991299 12695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6386869496b244cf9786f4f0998e0d1c, State: Initialized, Role: FOLLOWER
Aug 12 05:29:18 dist-test-slave-zpfg krb5kdc[12610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:18.991439 12695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6386869496b244cf9786f4f0998e0d1c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44909 } }
I20260812 05:29:18.991534 12695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:18.991581 12695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:18.991637 12695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:18.992272 12695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6386869496b244cf9786f4f0998e0d1c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44909 } }
I20260812 05:29:18.992395 12695 leader_election.cc:302] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6386869496b244cf9786f4f0998e0d1c; no voters:
I20260812 05:29:18.992631 12695 leader_election.cc:288] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:18.992641 12700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:18.992798 12700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [term 1 LEADER]: Becoming Leader. State: Replica: 6386869496b244cf9786f4f0998e0d1c, State: Running, Role: LEADER
I20260812 05:29:18.992898 12700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6386869496b244cf9786f4f0998e0d1c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44909 } }
I20260812 05:29:18.992961 12695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:18.993233 12700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6386869496b244cf9786f4f0998e0d1c. Latest consensus state: current_term: 1 leader_uuid: "6386869496b244cf9786f4f0998e0d1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6386869496b244cf9786f4f0998e0d1c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44909 } } }
I20260812 05:29:18.993314 12700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:18.993500 12701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6386869496b244cf9786f4f0998e0d1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6386869496b244cf9786f4f0998e0d1c" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 44909 } } }
I20260812 05:29:18.993659 12701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:18.993867 12704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:18.994530 12704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:18.996291 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:18.983677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47662 (local address 127.24.66.126:44909)
0812 05:29:18.983901 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:18.983912 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:18.983973 (+ 61us) server_negotiation.cc:407] Connection header received
0812 05:29:18.984894 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:18.984903 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:18.984974 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:18.985115 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:18.986206 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:18.987347 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:18.988520 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:18.989051 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:18.989192 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:18.992095 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:18.992099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:18.992099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:18.992110 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:18.994423 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:18.994989 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:18.994993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:18.994993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:18.995108 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:18.995369 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:18.995370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:18.995370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:18.995612 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:18.995801 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:18.995934 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 05:29:18.996079 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":138,"threads_started":1}
I20260812 05:29:18.998889 12704 catalog_manager.cc:1347] Generated new cluster ID: f3de26056e4b4c9e9124261cf8c67341
I20260812 05:29:18.998942 12704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:19.011623 12704 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:19.012424 12704 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:19.016765 12704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c: Generated new TSK 0
I20260812 05:29:19.016994 12704 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.66.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
Entry for principal kudu/127.24.66.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab.
I20260812 05:29:19.053738 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44909
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:19.158454 12725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:19.158675 12725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:19.158701 12725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:19.159893 12725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:19.159937 12725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:19.159989 12725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:29:19.161556 12725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.65.keytab
--principal=kudu/127.24.66.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:19.161824 12725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:19.162074 12725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:19.167464 12731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:19.167459 12730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:19.167518 12733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:19.167865 12725 server_base.cc:1034] running on GCE node
I20260812 05:29:19.168121 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:19.168399 12725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:19.169646 12725 hybrid_clock.cc:648] HybridClock initialized: now 1786512559169640 us; error 40 us; skew 500 ppm
Aug 12 05:29:19 dist-test-slave-zpfg krb5kdc[12610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512559, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:19.172789 12725 init.cc:377] Logged in from keytab as kudu/127.24.66.65@KRBTEST.COM (short username kudu)
I20260812 05:29:19.173353 12725 webserver.cc:460] Webserver started at http://127.24.66.65:38143/ using document root <none> and password file <none>
I20260812 05:29:19.173626 12725 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:19.173707 12725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:19.173880 12725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:19.175169 12725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "a8006e6111274c7d91cd74bb5cb5e80f"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "d3693efb131038bc1786162e0595542d"
server_key_iv: "6bdcc1437e00859fc30f5fc3a4767d80"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.175590 12725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "a8006e6111274c7d91cd74bb5cb5e80f"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "d3693efb131038bc1786162e0595542d"
server_key_iv: "6bdcc1437e00859fc30f5fc3a4767d80"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.177178 12725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 05:29:19.178097 12740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:19.178303 12725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:19.178439 12725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "a8006e6111274c7d91cd74bb5cb5e80f"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "d3693efb131038bc1786162e0595542d"
server_key_iv: "6bdcc1437e00859fc30f5fc3a4767d80"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.178521 12725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:19.199955 12725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:19.202538 12725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:19.202724 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:19.203001 12725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:19.203341 12725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:19.203399 12725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:19.203440 12725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:19.203469 12725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:19.209601 12725 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:34023
I20260812 05:29:19.209653 12860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:34023 every 8 connection(s)
I20260812 05:29:19.210003 12725 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:29:19.211035 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 12725
I20260812 05:29:19.211158 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:29:19.211396 24841 external_mini_cluster.cc:1428] Setting key f94314d1393a12963dac3c042fbf7e07
Aug 12 05:29:19 dist-test-slave-zpfg krb5kdc[12610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512559, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.65@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.66.66@KRBTEST.COM; defaulting to no policy
I20260812 05:29:19.221464 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.210859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:54581 (local address 127.24.66.126:44909)
0812 05:29:19.210937 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:19.210939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:19.211237 (+ 298us) server_negotiation.cc:407] Connection header received
0812 05:29:19.211944 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:19.211945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:19.211956 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:19.212025 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:19.213493 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.214027 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.215161 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.215504 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.215587 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.218235 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:19.218237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:19.218237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:19.218246 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:19.219928 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:19.220391 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:19.220393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:19.220393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:19.220460 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:19.220724 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:19.220725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:19.220725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:19.220897 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:19.220957 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:19.221254 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 05:29:19.221363 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:29:19.221653 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.210793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:44909 (local address 127.24.66.65:54581)
0812 05:29:19.211079 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:19.211131 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:19.211845 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:19.212113 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:19.212118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:19.213063 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:19.213377 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.213384 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.214138 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.214139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.215062 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.215065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.215598 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.215599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.215731 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.216352 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:19.216361 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:19.218129 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:19.220038 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:19.220047 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:19.220050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:19.220296 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:19.220528 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:19.220529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:19.220529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:19.220652 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:19.220961 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:19.220968 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:19.221117 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 05:29:19.221229 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 05:29:19.221408 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":89,"threads_started":1}
I20260812 05:29:19.222208 12861 heartbeater.cc:347] Connected to a master server at 127.24.66.126:44909
I20260812 05:29:19.222301 12861 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:19.222539 12861 heartbeater.cc:511] Master 127.24.66.126:44909 requested a full tablet report, sending...
I20260812 05:29:19.223181 12657 ts_manager.cc:194] Registered new tserver with Master: a8006e6111274c7d91cd74bb5cb5e80f (127.24.66.65:34023)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.66@KRBTEST.COM" created.
I20260812 05:29:19.223956 12657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.65@KRBTEST.COM'} at 127.24.66.65:54581
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
Entry for principal kudu/127.24.66.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab.
I20260812 05:29:19.241032 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44909
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:19.328279 12867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:19.328510 12867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:19.328534 12867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:19.329764 12867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:19.329814 12867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:19.329842 12867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:29:19.331410 12867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.66.keytab
--principal=kudu/127.24.66.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:19.331678 12867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:19.331933 12867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:29:19.337399 12867 server_base.cc:1034] running on GCE node
W20260812 05:29:19.337420 12877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:19.337533 12874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:19.337445 12873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:19.337837 12867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:19.338056 12867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:19.339228 12867 hybrid_clock.cc:648] HybridClock initialized: now 1786512559339218 us; error 39 us; skew 500 ppm
Aug 12 05:29:19 dist-test-slave-zpfg krb5kdc[12610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512559, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:19.341907 12867 init.cc:377] Logged in from keytab as kudu/127.24.66.66@KRBTEST.COM (short username kudu)
I20260812 05:29:19.342545 12867 webserver.cc:460] Webserver started at http://127.24.66.66:38935/ using document root <none> and password file <none>
I20260812 05:29:19.342868 12867 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:19.342950 12867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:19.343122 12867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:19.344426 12867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "d3a6d721d94a43eba0de3e5b55a2ad82"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "3d8cc6ec78c9f276f4a144a057b498bf"
server_key_iv: "31ea2fd6fb4632be310165a021223e16"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.344846 12867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "d3a6d721d94a43eba0de3e5b55a2ad82"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "3d8cc6ec78c9f276f4a144a057b498bf"
server_key_iv: "31ea2fd6fb4632be310165a021223e16"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.346561 12867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 05:29:19.347594 12884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:19.347831 12867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:19.347958 12867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "d3a6d721d94a43eba0de3e5b55a2ad82"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "3d8cc6ec78c9f276f4a144a057b498bf"
server_key_iv: "31ea2fd6fb4632be310165a021223e16"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.348040 12867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:19.364578 12867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:19.367506 12867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:19.367684 12867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:19.367976 12867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:19.368377 12867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:19.368438 12867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:19.368474 12867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:19.368501 12867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:19.375036 12867 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:35137
I20260812 05:29:19.375072 13001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:35137 every 8 connection(s)
I20260812 05:29:19.375430 12867 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:29:19.377409 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 12867
I20260812 05:29:19.377547 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:29:19.377744 24841 external_mini_cluster.cc:1428] Setting key 17a6ecc652e3d85cde8b6e8a7d9eb295
Aug 12 05:29:19 dist-test-slave-zpfg krb5kdc[12610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512559, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.66@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:19.386895 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.376201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:59937 (local address 127.24.66.126:44909)
0812 05:29:19.376282 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:19.376284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:19.376587 (+ 303us) server_negotiation.cc:407] Connection header received
0812 05:29:19.377190 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:19.377213 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:19.377227 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:19.377423 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:19.378712 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.379261 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.380638 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.380863 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.380924 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.383470 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:19.383471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:19.383472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:19.383480 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:19.385190 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:19.385713 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:19.385715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:19.385716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:19.385788 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:19.386065 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:19.386066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:19.386066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:19.386235 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:19.386328 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:19.386674 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 05:29:19.386781 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:29:19.386970 13004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.376185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:44909 (local address 127.24.66.66:59937)
0812 05:29:19.376429 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:19.376468 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:19.377067 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:19.377511 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:19.377514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:19.378287 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:19.378592 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.378597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.379455 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.379456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.380490 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.380493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.380958 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.380959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.381115 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.381795 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:19.381808 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:19.383359 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:19.385327 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:19.385359 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:19.385362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:19.385618 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:19.385877 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:19.385878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:19.385878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:19.385992 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:19.386320 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:19.386324 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:19.386526 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 05:29:19.386621 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 05:29:19.386759 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":159,"threads_started":1}
I20260812 05:29:19.387534 13002 heartbeater.cc:347] Connected to a master server at 127.24.66.126:44909
I20260812 05:29:19.387660 13002 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:19.387878 13002 heartbeater.cc:511] Master 127.24.66.126:44909 requested a full tablet report, sending...
I20260812 05:29:19.388343 12657 ts_manager.cc:194] Registered new tserver with Master: d3a6d721d94a43eba0de3e5b55a2ad82 (127.24.66.66:35137)
I20260812 05:29:19.388741 12657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.66@KRBTEST.COM'} at 127.24.66.66:59937
WARNING: no policy specified for kudu/127.24.66.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
Entry for principal kudu/127.24.66.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab.
I20260812 05:29:19.409211 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44909
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:19.495450 13007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:19.495676 13007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:19.495703 13007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:19.497319 13007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:19.497401 13007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:19.497448 13007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:29:19.499318 13007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.67.keytab
--principal=kudu/127.24.66.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:44909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:19.499624 13007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:19.499891 13007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:19.505326 13013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:19.505393 13015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:19.505551 13007 server_base.cc:1034] running on GCE node
W20260812 05:29:19.505357 13012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:19.505843 13007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:19.506160 13007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:19.507421 13007 hybrid_clock.cc:648] HybridClock initialized: now 1786512559507382 us; error 49 us; skew 500 ppm
Aug 12 05:29:19 dist-test-slave-zpfg krb5kdc[12610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512559, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:19.510070 13007 init.cc:377] Logged in from keytab as kudu/127.24.66.67@KRBTEST.COM (short username kudu)
I20260812 05:29:19.510600 13007 webserver.cc:460] Webserver started at http://127.24.66.67:45693/ using document root <none> and password file <none>
I20260812 05:29:19.510802 13007 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:19.510846 13007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:19.510943 13007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:19.512005 13007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "1373bbca0a354815a97fca04ce2cba6b"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "468fff837eed723558c2fd87e705d346"
server_key_iv: "85b95e954bba491a13439aa153411be1"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.512331 13007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "1373bbca0a354815a97fca04ce2cba6b"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "468fff837eed723558c2fd87e705d346"
server_key_iv: "85b95e954bba491a13439aa153411be1"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.513669 13007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:19.514504 13023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:19.514654 13007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:19.514731 13007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "1373bbca0a354815a97fca04ce2cba6b"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "468fff837eed723558c2fd87e705d346"
server_key_iv: "85b95e954bba491a13439aa153411be1"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.514804 13007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:19.533205 13007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:19.535728 13007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:19.535857 13007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:19.536065 13007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:19.536419 13007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:19.536455 13007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:19.536480 13007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:19.536526 13007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:19.542809 13007 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:35559
I20260812 05:29:19.542905 13147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:35559 every 8 connection(s)
I20260812 05:29:19.543259 13007 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:29:19.545670 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 13007
I20260812 05:29:19.545792 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:29:19.545996 24841 external_mini_cluster.cc:1428] Setting key 6ca5d5a954c7581f72e8d7adcd2ff96c
Aug 12 05:29:19 dist-test-slave-zpfg krb5kdc[12610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512559, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.67@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:19.554685 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.544143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:43439 (local address 127.24.66.126:44909)
0812 05:29:19.544255 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:19.544258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:19.544472 (+ 214us) server_negotiation.cc:407] Connection header received
0812 05:29:19.545127 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:19.545128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:19.545142 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:19.545204 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:19.546781 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.547455 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.549050 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.549387 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.549453 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.551955 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:19.551957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:19.551957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:19.551964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:19.553385 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:19.553806 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:19.553807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:19.553807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:19.553859 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:19.554083 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:19.554083 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:19.554084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:19.554187 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:19.554238 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:19.554548 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 05:29:19.554616 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 05:29:19.554921 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.544115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:44909 (local address 127.24.66.67:43439)
0812 05:29:19.544323 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:19.544374 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:19.544990 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:19.545325 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:19.545329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:19.546303 (+ 974us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:19.546660 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.546668 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.547569 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.547571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.548915 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.548919 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.549464 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.549465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.549585 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.550152 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:19.550164 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:19.551860 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:19.553470 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:19.553477 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:19.553479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:19.553727 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:19.553919 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:19.553920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:19.553921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:19.554028 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:19.554247 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:19.554251 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:19.554428 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 05:29:19.554517 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 05:29:19.554646 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":89,"threads_started":1}
I20260812 05:29:19.555538 13148 heartbeater.cc:347] Connected to a master server at 127.24.66.126:44909
I20260812 05:29:19.555672 13148 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:19.555879 13148 heartbeater.cc:511] Master 127.24.66.126:44909 requested a full tablet report, sending...
I20260812 05:29:19.556265 12657 ts_manager.cc:194] Registered new tserver with Master: 1373bbca0a354815a97fca04ce2cba6b (127.24.66.67:35559)
I20260812 05:29:19.556607 12657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.66.67@KRBTEST.COM'} at 127.24.66.67:43439
I20260812 05:29:19.557833 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:29:19.566123 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.559411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47666 (local address 127.24.66.126:44909)
0812 05:29:19.559487 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:19.559489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:19.559524 (+ 35us) server_negotiation.cc:407] Connection header received
0812 05:29:19.559602 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:19.559603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:19.559611 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:19.559657 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:19.560242 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.560729 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.561678 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.561879 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.561935 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.563441 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:19.563443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:19.563443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:19.563449 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:19.564924 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:19.565271 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:19.565272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:19.565272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:19.565323 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:19.565493 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:19.565494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:19.565494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:19.565638 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:19.565717 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:19.565983 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 05:29:19.566052 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 05:29:19.568073 12657 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:29:19.569787 12657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:29:19.577481 13160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.572400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:34023 (local address 127.0.0.1:43896)
0812 05:29:19.572850 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:19.572862 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:19.572924 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:19.573421 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:19.573423 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:19.573425 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:19.573463 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.573464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.574719 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.574720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.576248 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.576250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.577097 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.577099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.577197 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.577218 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 05:29:19.577321 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 05:29:19.577382 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":64,"thread_start_us":97,"threads_started":1}
I20260812 05:29:19.577831 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.572649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43896 (local address 127.24.66.65:34023)
0812 05:29:19.572885 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:19.572891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:19.572906 (+ 15us) server_negotiation.cc:407] Connection header received
0812 05:29:19.573012 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:19.573014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:19.573028 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:19.573108 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:19.573530 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.574612 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.576339 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.576993 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.577052 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.577093 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:19.577349 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 05:29:19.577397 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":97,"threads_started":1}
I20260812 05:29:19.578209 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.572352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:35559 (local address 127.0.0.1:40296)
0812 05:29:19.572475 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:19.572519 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:19.572588 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:19.573256 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:19.573258 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:19.573260 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:19.573337 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.573339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.574720 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.574720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.576236 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.576240 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.577993 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.577994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.578066 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.578070 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:29:19.578094 (+ 24us) client_negotiation.cc:240] Negotiation successful
0812 05:29:19.578132 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":115,"thread_start_us":75,"threads_started":1}
I20260812 05:29:19.578555 13162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.572537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.24.66.67:35559)
0812 05:29:19.573088 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:19.573094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:19.573105 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:29:19.573147 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:19.573150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:19.573166 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:19.573247 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:19.573404 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.574605 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.577127 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.577899 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.578374 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.578444 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:19.578465 (+ 21us) server_negotiation.cc:299] Negotiation successful
0812 05:29:19.578499 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":99,"threads_started":1}
I20260812 05:29:19.578882 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.572681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45866 (local address 127.24.66.66:35137)
0812 05:29:19.572885 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:19.572891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:19.572906 (+ 15us) server_negotiation.cc:407] Connection header received
0812 05:29:19.572995 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:19.572997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:19.573014 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:19.573108 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:19.574738 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.575664 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.576736 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.577787 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.578497 (+ 710us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.578567 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:19.578741 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 05:29:19.578790 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
I20260812 05:29:19.578868 12792 tablet_service.cc:1467] Processing CreateTablet for tablet a05f675c782b465d83061c860f0b92c7 (DEFAULT_TABLE table=test-table [id=a7c66d4d9766468a981bf22f1cc710ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:19.579270 12792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a05f675c782b465d83061c860f0b92c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:19.582810 13076 tablet_service.cc:1467] Processing CreateTablet for tablet a05f675c782b465d83061c860f0b92c7 (DEFAULT_TABLE table=test-table [id=a7c66d4d9766468a981bf22f1cc710ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:19.583105 13076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a05f675c782b465d83061c860f0b92c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:19.585870 13167 tablet_bootstrap.cc:492] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b: Bootstrap starting.
I20260812 05:29:19.586045 13166 tablet_bootstrap.cc:492] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f: Bootstrap starting.
I20260812 05:29:19.578217 13161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.572400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:35137 (local address 127.0.0.1:45866)
0812 05:29:19.572861 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:19.572871 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:19.572921 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:19.573245 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:19.573249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:19.573251 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:19.573338 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.573341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.575767 (+ 2426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.575768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.576628 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.576631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.577917 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.577918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.578013 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.578020 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 05:29:19.578060 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 05:29:19.578109 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":14,"thread_start_us":88,"threads_started":1}
I20260812 05:29:19.586800 13167 tablet_bootstrap.cc:654] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:19.586800 13166 tablet_bootstrap.cc:654] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:19.587050 12932 tablet_service.cc:1467] Processing CreateTablet for tablet a05f675c782b465d83061c860f0b92c7 (DEFAULT_TABLE table=test-table [id=a7c66d4d9766468a981bf22f1cc710ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:19.587416 12932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a05f675c782b465d83061c860f0b92c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:19.587466 13167 log.cc:824] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:19.587466 13166 log.cc:824] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:19.588336 13166 tablet_bootstrap.cc:492] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f: No bootstrap required, opened a new log
I20260812 05:29:19.588416 13166 ts_tablet_manager.cc:1397] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 05:29:19.588840 13167 tablet_bootstrap.cc:492] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b: No bootstrap required, opened a new log
I20260812 05:29:19.588912 13167 ts_tablet_manager.cc:1397] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:29:19.590576 13170 tablet_bootstrap.cc:492] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: Bootstrap starting.
I20260812 05:29:19.590566 13166 raft_consensus.cc:348] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1373bbca0a354815a97fca04ce2cba6b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35559 } } peers { permanent_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34023 } } peers { permanent_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35137 } }
I20260812 05:29:19.590768 13166 raft_consensus.cc:374] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:19.590821 13166 raft_consensus.cc:729] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8006e6111274c7d91cd74bb5cb5e80f, State: Initialized, Role: FOLLOWER
I20260812 05:29:19.590874 13167 raft_consensus.cc:348] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1373bbca0a354815a97fca04ce2cba6b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35559 } } peers { permanent_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34023 } } peers { permanent_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35137 } }
I20260812 05:29:19.590989 13166 consensus_queue.cc:260] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1373bbca0a354815a97fca04ce2cba6b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35559 } } peers { permanent_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34023 } } peers { permanent_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35137 } }
I20260812 05:29:19.591038 13167 raft_consensus.cc:374] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:19.591074 13167 raft_consensus.cc:729] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1373bbca0a354815a97fca04ce2cba6b, State: Initialized, Role: FOLLOWER
I20260812 05:29:19.591209 13167 consensus_queue.cc:260] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1373bbca0a354815a97fca04ce2cba6b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35559 } } peers { permanent_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34023 } } peers { permanent_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35137 } }
I20260812 05:29:19.591288 13166 ts_tablet_manager.cc:1428] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 05:29:19.591411 12861 heartbeater.cc:503] Master 127.24.66.126:44909 was elected leader, sending a full tablet report...
I20260812 05:29:19.591508 13170 tablet_bootstrap.cc:654] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:19.591781 13167 ts_tablet_manager.cc:1428] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:29:19.591954 13148 heartbeater.cc:503] Master 127.24.66.126:44909 was elected leader, sending a full tablet report...
I20260812 05:29:19.592254 13170 log.cc:824] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:19.593253 13170 tablet_bootstrap.cc:492] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: No bootstrap required, opened a new log
I20260812 05:29:19.593325 13170 ts_tablet_manager.cc:1397] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 05:29:19.595204 13170 raft_consensus.cc:348] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1373bbca0a354815a97fca04ce2cba6b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35559 } } peers { permanent_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34023 } } peers { permanent_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35137 } }
I20260812 05:29:19.595358 13170 raft_consensus.cc:374] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:19.595383 13170 raft_consensus.cc:729] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3a6d721d94a43eba0de3e5b55a2ad82, State: Initialized, Role: FOLLOWER
I20260812 05:29:19.595551 13170 consensus_queue.cc:260] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1373bbca0a354815a97fca04ce2cba6b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35559 } } peers { permanent_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34023 } } peers { permanent_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35137 } }
I20260812 05:29:19.595891 13170 ts_tablet_manager.cc:1428] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:29:19.595906 13002 heartbeater.cc:503] Master 127.24.66.126:44909 was elected leader, sending a full tablet report...
I20260812 05:29:19.621340 13171 raft_consensus.cc:482] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:29:19.621491 13171 raft_consensus.cc:504] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1373bbca0a354815a97fca04ce2cba6b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35559 } } peers { permanent_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34023 } } peers { permanent_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35137 } }
I20260812 05:29:19.621887 13171 leader_election.cc:288] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1373bbca0a354815a97fca04ce2cba6b (127.24.66.67:35559), d3a6d721d94a43eba0de3e5b55a2ad82 (127.24.66.66:35137)
I20260812 05:29:19.624990 13162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.622221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:46543 (local address 127.24.66.67:35559)
0812 05:29:19.622273 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:19.622276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:19.622470 (+ 194us) server_negotiation.cc:407] Connection header received
0812 05:29:19.622546 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:19.622547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:19.622556 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:19.622630 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:19.622798 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.623272 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.624242 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.624624 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.624708 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.624719 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:19.624878 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 05:29:19.624918 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 05:29:19.625010 13175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.622081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:35559 (local address 127.24.66.65:46543)
0812 05:29:19.622398 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:19.622408 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:19.622480 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:19.622635 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:19.622635 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:19.622637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:19.622720 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.622722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.623363 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.623363 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.624146 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.624148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.624722 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.624723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.624803 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.624807 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 05:29:19.624873 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 05:29:19.624912 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":16,"thread_start_us":130,"threads_started":1}
I20260812 05:29:19.625134 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.622209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:37701 (local address 127.24.66.66:35137)
0812 05:29:19.622270 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:19.622273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:19.622299 (+ 26us) server_negotiation.cc:407] Connection header received
0812 05:29:19.622378 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:19.622380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:19.622392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:19.622454 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:19.622643 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.623235 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.624309 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.624746 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.624844 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.624852 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:19.625015 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 05:29:19.625059 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 05:29:19.625157 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.622081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:35137 (local address 127.24.66.65:37701)
0812 05:29:19.622153 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:19.622167 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:19.622245 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:19.622471 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:19.622472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:19.622474 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:19.622556 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.622558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.623345 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.623347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.624204 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:19.624207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.624843 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.624844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:19.624932 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.624935 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:29:19.625010 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 05:29:19.625078 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":28}
I20260812 05:29:19.625428 13097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a05f675c782b465d83061c860f0b92c7" candidate_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1373bbca0a354815a97fca04ce2cba6b" is_pre_election: true
I20260812 05:29:19.625473 12953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a05f675c782b465d83061c860f0b92c7" candidate_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" is_pre_election: true
I20260812 05:29:19.625582 13097 raft_consensus.cc:2454] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8006e6111274c7d91cd74bb5cb5e80f in term 0.
I20260812 05:29:19.625667 12953 raft_consensus.cc:2454] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8006e6111274c7d91cd74bb5cb5e80f in term 0.
I20260812 05:29:19.625856 12741 leader_election.cc:302] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1373bbca0a354815a97fca04ce2cba6b, a8006e6111274c7d91cd74bb5cb5e80f; no voters:
W20260812 05:29:19.626034 13003 tablet.cc:2369] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 05:29:19.626016 13171 raft_consensus.cc:2781] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:29:19.626089 13171 raft_consensus.cc:482] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:29:19.626134 13171 raft_consensus.cc:3037] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:19.627020 13171 raft_consensus.cc:504] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1373bbca0a354815a97fca04ce2cba6b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35559 } } peers { permanent_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34023 } } peers { permanent_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35137 } }
I20260812 05:29:19.627204 13171 leader_election.cc:288] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [CANDIDATE]: Term 1 election: Requested vote from peers 1373bbca0a354815a97fca04ce2cba6b (127.24.66.67:35559), d3a6d721d94a43eba0de3e5b55a2ad82 (127.24.66.66:35137)
I20260812 05:29:19.627372 12953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a05f675c782b465d83061c860f0b92c7" candidate_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82"
I20260812 05:29:19.627384 13097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a05f675c782b465d83061c860f0b92c7" candidate_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1373bbca0a354815a97fca04ce2cba6b"
I20260812 05:29:19.627454 12953 raft_consensus.cc:3037] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:19.627480 13097 raft_consensus.cc:3037] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:19.628265 12953 raft_consensus.cc:2454] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8006e6111274c7d91cd74bb5cb5e80f in term 1.
I20260812 05:29:19.628372 13097 raft_consensus.cc:2454] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8006e6111274c7d91cd74bb5cb5e80f in term 1.
I20260812 05:29:19.628468 12742 leader_election.cc:302] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8006e6111274c7d91cd74bb5cb5e80f, d3a6d721d94a43eba0de3e5b55a2ad82; no voters:
I20260812 05:29:19.628561 13171 raft_consensus.cc:2781] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:19.628804 13171 raft_consensus.cc:686] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [term 1 LEADER]: Becoming Leader. State: Replica: a8006e6111274c7d91cd74bb5cb5e80f, State: Running, Role: LEADER
I20260812 05:29:19.628918 13171 consensus_queue.cc:237] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1373bbca0a354815a97fca04ce2cba6b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35559 } } peers { permanent_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34023 } } peers { permanent_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35137 } }
I20260812 05:29:19.629933 12657 catalog_manager.cc:5495] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f reported cstate change: term changed from 0 to 1, leader changed from <none> to a8006e6111274c7d91cd74bb5cb5e80f (127.24.66.65). New cstate: current_term: 1 leader_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1373bbca0a354815a97fca04ce2cba6b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8006e6111274c7d91cd74bb5cb5e80f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35137 } health_report { overall_health: UNKNOWN } } }
I20260812 05:29:19.649866 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.647500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43902 (local address 127.24.66.65:34023)
0812 05:29:19.647577 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:19.647580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:19.647588 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:29:19.647628 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:19.647628 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:19.647647 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:19.647708 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:19.647869 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.648334 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.649103 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.649308 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.649367 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.649507 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:19.649614 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:19.649676 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:19.649705 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 05:29:19.649723 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:29:19.652047 12953 raft_consensus.cc:1261] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82 [term 1 FOLLOWER]: Refusing update from remote peer a8006e6111274c7d91cd74bb5cb5e80f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:19.652153 13097 raft_consensus.cc:1261] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b [term 1 FOLLOWER]: Refusing update from remote peer a8006e6111274c7d91cd74bb5cb5e80f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:19.652484 13176 consensus_queue.cc:1035] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3a6d721d94a43eba0de3e5b55a2ad82" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 35137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:19.652777 13176 consensus_queue.cc:1035] T a05f675c782b465d83061c860f0b92c7 P a8006e6111274c7d91cd74bb5cb5e80f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1373bbca0a354815a97fca04ce2cba6b" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:19.657590 12656 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:19.657694 12656 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:19.658568 12656 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6386869496b244cf9786f4f0998e0d1c: Sending DeleteTablet for 3 replicas of tablet a05f675c782b465d83061c860f0b92c7
I20260812 05:29:19.659008 12932 tablet_service.cc:1514] Processing DeleteTablet for tablet a05f675c782b465d83061c860f0b92c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:19 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:45866
I20260812 05:29:19.659284 13189 tablet_replica.cc:330] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: stopping tablet replica
I20260812 05:29:19.659277 13076 tablet_service.cc:1514] Processing DeleteTablet for tablet a05f675c782b465d83061c860f0b92c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:19 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:40296
I20260812 05:29:19.659400 13189 raft_consensus.cc:2229] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:29:19.659507 13189 raft_consensus.cc:2258] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:19.659539 12792 tablet_service.cc:1514] Processing DeleteTablet for tablet a05f675c782b465d83061c860f0b92c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:19 UTC) from {username='kudu', principal='kudu/127.24.66.126@KRBTEST.COM'} at 127.0.0.1:43896
I20260812 05:29:19.659960 13189 ts_tablet_manager.cc:1905] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:19.660068 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 12725
I20260812 05:29:19.662854 13189 ts_tablet_manager.cc:1918] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:19.662945 13189 log.cc:1196] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/a05f675c782b465d83061c860f0b92c7
I20260812 05:29:19.663232 13189 ts_tablet_manager.cc:1939] T a05f675c782b465d83061c860f0b92c7 P d3a6d721d94a43eba0de3e5b55a2ad82: Deleting consensus metadata
I20260812 05:29:19.663667 12641 catalog_manager.cc:4841] TS d3a6d721d94a43eba0de3e5b55a2ad82 (127.24.66.66:35137): tablet a05f675c782b465d83061c860f0b92c7 (table test-table [id=a7c66d4d9766468a981bf22f1cc710ed]) successfully deleted
I20260812 05:29:19.665122 13191 tablet_replica.cc:330] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b: stopping tablet replica
I20260812 05:29:19.665217 13191 raft_consensus.cc:2229] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:29:19.665309 13191 raft_consensus.cc:2258] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:19.665843 13191 ts_tablet_manager.cc:1905] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:19.667898 13191 ts_tablet_manager.cc:1918] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:19.667984 13191 log.cc:1196] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/a05f675c782b465d83061c860f0b92c7
I20260812 05:29:19.668251 13191 ts_tablet_manager.cc:1939] T a05f675c782b465d83061c860f0b92c7 P 1373bbca0a354815a97fca04ce2cba6b: Deleting consensus metadata
I20260812 05:29:19.668730 12640 catalog_manager.cc:4841] TS 1373bbca0a354815a97fca04ce2cba6b (127.24.66.67:35559): tablet a05f675c782b465d83061c860f0b92c7 (table test-table [id=a7c66d4d9766468a981bf22f1cc710ed]) successfully deleted
W20260812 05:29:19.670797 12643 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:34023 (error 108)
I20260812 05:29:19.670993 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 12867
I20260812 05:29:19.672892 13161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.672731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:34023 (local address 127.0.0.1:43916)
0812 05:29:19.672796 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:19.672846 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:34023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 05:29:19.673141 12643 catalog_manager.cc:4584] TS a8006e6111274c7d91cd74bb5cb5e80f (127.24.66.65:34023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a05f675c782b465d83061c860f0b92c7: Network error: Client connection negotiation failed: client connection to 127.24.66.65:34023: connect: Connection refused (error 111)
I20260812 05:29:19.677677 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 13007
I20260812 05:29:19.683789 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 12624
2026-08-12T05:29:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2090 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T05:29:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:19Z Disabled control of system clock
I20260812 05:29:19.715241 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:43075
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:43075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:29:19.810066 13197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:19.810300 13197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:19.810323 13197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:19.811610 13197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:19.811656 13197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:19.811671 13197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:19.811682 13197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:19.811695 13197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:19.813002 13197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:43075
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:43075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:19.813210 13197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:19.813483 13197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:19.818645 13202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:19.818797 13197 server_base.cc:1034] running on GCE node
W20260812 05:29:19.818645 13205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:19.818684 13203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:19.819198 13197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:19.819550 13197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:19.820708 13197 hybrid_clock.cc:648] HybridClock initialized: now 1786512559820685 us; error 40 us; skew 500 ppm
I20260812 05:29:19.822095 13197 webserver.cc:460] Webserver started at http://127.24.66.126:44709/ using document root <none> and password file <none>
I20260812 05:29:19.822455 13197 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:19.822533 13197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:19.822677 13197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:19.823783 13197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "01291b952761452ba8ff2b6fa05a0804"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "24ca4b5b748caf1039fe5af8a93f162a"
server_key_iv: "2bc1a324f41cf4367cd3f2cc56b59de8"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.824174 13197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "01291b952761452ba8ff2b6fa05a0804"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "24ca4b5b748caf1039fe5af8a93f162a"
server_key_iv: "2bc1a324f41cf4367cd3f2cc56b59de8"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.825582 13197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:19.826512 13211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:19.826671 13197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:19.826767 13197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "01291b952761452ba8ff2b6fa05a0804"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "24ca4b5b748caf1039fe5af8a93f162a"
server_key_iv: "2bc1a324f41cf4367cd3f2cc56b59de8"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.826828 13197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:19.849013 13197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:19.849444 13197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:19.849620 13197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:19.853948 13197 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:43075
I20260812 05:29:19.853982 13270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:43075 every 8 connection(s)
I20260812 05:29:19.854410 13197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:19.855707 13271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:19.858174 13271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804: Bootstrap starting.
I20260812 05:29:19.858851 13271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:19.859117 13271 log.cc:824] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:19.859936 13271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804: No bootstrap required, opened a new log
I20260812 05:29:19.860634 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 13197
I20260812 05:29:19.860728 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:19.860935 24841 external_mini_cluster.cc:1428] Setting key 0ee061715ea6853a13d470d283153c00
I20260812 05:29:19.862380 13271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01291b952761452ba8ff2b6fa05a0804" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43075 } }
I20260812 05:29:19.862581 13271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:19.862636 13271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01291b952761452ba8ff2b6fa05a0804, State: Initialized, Role: FOLLOWER
I20260812 05:29:19.862771 13271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01291b952761452ba8ff2b6fa05a0804" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43075 } }
I20260812 05:29:19.862877 13271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:19.862922 13271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:19.862972 13271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:19.863566 13271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01291b952761452ba8ff2b6fa05a0804" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43075 } }
I20260812 05:29:19.863700 13271 leader_election.cc:302] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01291b952761452ba8ff2b6fa05a0804; no voters:
I20260812 05:29:19.863941 13271 leader_election.cc:288] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:19.863974 13278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:19.864089 13278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [term 1 LEADER]: Becoming Leader. State: Replica: 01291b952761452ba8ff2b6fa05a0804, State: Running, Role: LEADER
I20260812 05:29:19.864179 13278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01291b952761452ba8ff2b6fa05a0804" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43075 } }
I20260812 05:29:19.864256 13271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:19.864542 13279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01291b952761452ba8ff2b6fa05a0804" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01291b952761452ba8ff2b6fa05a0804" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43075 } } }
I20260812 05:29:19.864641 13279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:19.864858 13280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01291b952761452ba8ff2b6fa05a0804. Latest consensus state: current_term: 1 leader_uuid: "01291b952761452ba8ff2b6fa05a0804" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01291b952761452ba8ff2b6fa05a0804" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43075 } } }
I20260812 05:29:19.864943 13280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:19.864890 13284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:19.865626 13284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:19.867992 13284 catalog_manager.cc:1347] Generated new cluster ID: f2a34684e59249ffab26fae2b1cf309e
I20260812 05:29:19.868050 13284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:19.868152 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:19.861540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33258 (local address 127.24.66.126:43075)
0812 05:29:19.861785 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:19.861795 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:19.861833 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:29:19.862298 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:19.862308 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:19.862419 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:19.862574 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:19.862848 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.864496 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.865685 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:19.866249 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:19.866404 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:19.867289 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:19.867293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:19.867294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:19.867520 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:19.867565 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:19.867588 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:19.867672 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:19.867741 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 05:29:19.867829 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
I20260812 05:29:19.872677 13284 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:19.873226 13284 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:19.877931 13284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804: Generated new TSK 0
I20260812 05:29:19.878103 13284 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:19.882793 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:43075
--builtin_ntp_servers=127.24.66.84:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 05:29:19.979568 13299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:19.979786 13299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:19.979809 13299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:19.981009 13299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:19.981050 13299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:19.981076 13299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20260812 05:29:19.982548 13299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:43075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:19.982812 13299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:19.983192 13299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:19.988518 13304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:19.988613 13305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:19.988518 13307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:19.988729 13299 server_base.cc:1034] running on GCE node
I20260812 05:29:19.989077 13299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:19.989344 13299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:19.990510 13299 hybrid_clock.cc:648] HybridClock initialized: now 1786512559990492 us; error 38 us; skew 500 ppm
I20260812 05:29:19.991855 13299 webserver.cc:460] Webserver started at http://127.24.66.65:46015/ using document root <none> and password file <none>
I20260812 05:29:19.992131 13299 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:19.992206 13299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:19.992408 13299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:19.993511 13299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/data/instance:
uuid: "cadc342dda154e169ee6ebe917eecfdc"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "614b4b3589c7ecba72f85d2ac3e8390e"
server_key_iv: "118a4df3ccabe0f291c465ee5ac5d5d0"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.993903 13299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance:
uuid: "cadc342dda154e169ee6ebe917eecfdc"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "614b4b3589c7ecba72f85d2ac3e8390e"
server_key_iv: "118a4df3ccabe0f291c465ee5ac5d5d0"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.995515 13299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:29:19.996441 13313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:19.996748 13299 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:19.996843 13299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/wal
uuid: "cadc342dda154e169ee6ebe917eecfdc"
format_stamp: "Formatted at 2026-08-12 05:29:19 on dist-test-slave-zpfg"
server_key: "614b4b3589c7ecba72f85d2ac3e8390e"
server_key_iv: "118a4df3ccabe0f291c465ee5ac5d5d0"
server_key_version: "encryptionkey@0"
I20260812 05:29:19.996923 13299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:20.017249 13299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:20.017673 13299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:20.017829 13299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:20.018105 13299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:20.018509 13299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:20.018554 13299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:20.018585 13299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:20.018618 13299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:20.025940 13299 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:33299
I20260812 05:29:20.025980 13438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:33299 every 8 connection(s)
I20260812 05:29:20.026450 13299 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/data/info.pb
I20260812 05:29:20.029852 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 13299
I20260812 05:29:20.030004 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-0/wal/instance
I20260812 05:29:20.030296 24841 external_mini_cluster.cc:1428] Setting key 4b61611fa3edc69058d27700e9c21324
I20260812 05:29:20.031457 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:43075
--builtin_ntp_servers=127.24.66.84:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 05:29:20.032440 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.027339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.65:46821 (local address 127.24.66.126:43075)
0812 05:29:20.027424 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:20.027427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:20.027647 (+ 220us) server_negotiation.cc:407] Connection header received
0812 05:29:20.028138 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:20.028139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:20.028151 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:20.028206 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:20.028873 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.029382 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.030756 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.030987 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.031052 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.032101 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:20.032103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:20.032103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:20.032109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:20.032127 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:20.032132 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:20.032192 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:20.032284 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 05:29:20.032318 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 05:29:20.032578 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.027286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:43075 (local address 127.24.66.65:46821)
0812 05:29:20.027518 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:20.027557 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:20.028050 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:20.028263 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:20.028267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:20.028275 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:20.028793 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.028798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.029491 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.029497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.030586 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.030589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.031089 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.031090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.031257 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.031945 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:29:20.031945 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:20.031976 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:29:20.032000 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:20.032199 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:20.032204 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:20.032207 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:29:20.032263 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 05:29:20.032363 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260812 05:29:20.033152 13441 heartbeater.cc:347] Connected to a master server at 127.24.66.126:43075
I20260812 05:29:20.033270 13441 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:20.033468 13441 heartbeater.cc:511] Master 127.24.66.126:43075 requested a full tablet report, sending...
I20260812 05:29:20.034080 13231 ts_manager.cc:194] Registered new tserver with Master: cadc342dda154e169ee6ebe917eecfdc (127.24.66.65:33299)
I20260812 05:29:20.034776 13231 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.65:46821
W20260812 05:29:20.118554 13444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:20.118789 13444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:20.118815 13444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:20.120033 13444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:20.120077 13444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:20.120103 13444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20260812 05:29:20.121485 13444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:43075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:20.121770 13444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:20.122093 13444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:20.127456 13450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:20.127465 13452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:20.127480 13449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:20.127830 13444 server_base.cc:1034] running on GCE node
I20260812 05:29:20.128014 13444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:20.128222 13444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:20.129390 13444 hybrid_clock.cc:648] HybridClock initialized: now 1786512560129361 us; error 41 us; skew 500 ppm
I20260812 05:29:20.130798 13444 webserver.cc:460] Webserver started at http://127.24.66.66:46727/ using document root <none> and password file <none>
I20260812 05:29:20.131078 13444 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:20.131129 13444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:20.131310 13444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:20.132555 13444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/data/instance:
uuid: "564b1f8cb7c548a883cbecb92882de16"
format_stamp: "Formatted at 2026-08-12 05:29:20 on dist-test-slave-zpfg"
server_key: "ef79779532fa8d42779eb3d6b355e1c1"
server_key_iv: "e44105e9689d311ab383bbace75b6e9e"
server_key_version: "encryptionkey@0"
I20260812 05:29:20.132932 13444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance:
uuid: "564b1f8cb7c548a883cbecb92882de16"
format_stamp: "Formatted at 2026-08-12 05:29:20 on dist-test-slave-zpfg"
server_key: "ef79779532fa8d42779eb3d6b355e1c1"
server_key_iv: "e44105e9689d311ab383bbace75b6e9e"
server_key_version: "encryptionkey@0"
I20260812 05:29:20.134407 13444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260812 05:29:20.135212 13458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:20.135409 13444 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:20.135504 13444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/wal
uuid: "564b1f8cb7c548a883cbecb92882de16"
format_stamp: "Formatted at 2026-08-12 05:29:20 on dist-test-slave-zpfg"
server_key: "ef79779532fa8d42779eb3d6b355e1c1"
server_key_iv: "e44105e9689d311ab383bbace75b6e9e"
server_key_version: "encryptionkey@0"
I20260812 05:29:20.135583 13444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:20.149434 13444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:20.149931 13444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:20.150120 13444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:20.150478 13444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:20.150872 13444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:20.150915 13444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:20.150939 13444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:20.150990 13444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:20.157171 13444 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:38011
I20260812 05:29:20.157214 13575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:38011 every 8 connection(s)
I20260812 05:29:20.157567 13444 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/data/info.pb
I20260812 05:29:20.157866 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 13444
I20260812 05:29:20.157999 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/wal/instance
I20260812 05:29:20.158262 24841 external_mini_cluster.cc:1428] Setting key c5535dbf18d0a7685db499fc997fcbeb
I20260812 05:29:20.159461 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:43075
--builtin_ntp_servers=127.24.66.84:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 05:29:20.164597 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.158472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.66:38291 (local address 127.24.66.126:43075)
0812 05:29:20.158559 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:20.158562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:20.158871 (+ 309us) server_negotiation.cc:407] Connection header received
0812 05:29:20.159479 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:20.159481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:20.159494 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:20.159570 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:20.160337 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.161102 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.162773 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.163149 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.163227 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.164318 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:20.164321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:20.164322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:20.164330 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:20.164346 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:20.164350 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:20.164396 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:20.164503 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 05:29:20.164530 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 05:29:20.165040 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.158457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:43075 (local address 127.24.66.66:38291)
0812 05:29:20.158743 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:20.158782 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:20.159370 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:20.159684 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:20.159688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:20.159698 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:20.160234 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.160239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.161228 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.161229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.162593 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.162596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.163241 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.163242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.163413 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.164141 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:29:20.164142 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:20.164185 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:29:20.164221 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:20.164415 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:20.164422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:20.164425 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:29:20.164503 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 05:29:20.164695 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":113,"threads_started":1}
I20260812 05:29:20.165588 13576 heartbeater.cc:347] Connected to a master server at 127.24.66.126:43075
I20260812 05:29:20.165709 13576 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:20.165962 13576 heartbeater.cc:511] Master 127.24.66.126:43075 requested a full tablet report, sending...
I20260812 05:29:20.166419 13231 ts_manager.cc:194] Registered new tserver with Master: 564b1f8cb7c548a883cbecb92882de16 (127.24.66.66:38011)
I20260812 05:29:20.166992 13231 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:38291
W20260812 05:29:20.247409 13579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:20.247643 13579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:20.247668 13579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:20.248842 13579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:20.248886 13579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:20.248912 13579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20260812 05:29:20.250243 13579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:43075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:20.250494 13579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:20.250779 13579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:20.256479 13585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:20.256470 13584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:20.256524 13579 server_base.cc:1034] running on GCE node
W20260812 05:29:20.256470 13587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:20.257009 13579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:20.257237 13579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:20.258437 13579 hybrid_clock.cc:648] HybridClock initialized: now 1786512560258410 us; error 41 us; skew 500 ppm
I20260812 05:29:20.259918 13579 webserver.cc:460] Webserver started at http://127.24.66.67:43023/ using document root <none> and password file <none>
I20260812 05:29:20.260205 13579 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:20.260260 13579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:20.260442 13579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:20.261562 13579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/data/instance:
uuid: "34c331b30ad84216b035926775ab3fc4"
format_stamp: "Formatted at 2026-08-12 05:29:20 on dist-test-slave-zpfg"
server_key: "31fddb09758e4a8d57ac4a7517b660b1"
server_key_iv: "758ff69736a5a9cd9e153ef06606bee8"
server_key_version: "encryptionkey@0"
I20260812 05:29:20.261945 13579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance:
uuid: "34c331b30ad84216b035926775ab3fc4"
format_stamp: "Formatted at 2026-08-12 05:29:20 on dist-test-slave-zpfg"
server_key: "31fddb09758e4a8d57ac4a7517b660b1"
server_key_iv: "758ff69736a5a9cd9e153ef06606bee8"
server_key_version: "encryptionkey@0"
I20260812 05:29:20.263427 13579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:29:20.264453 13594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:20.264689 13579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:20.264796 13579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/wal
uuid: "34c331b30ad84216b035926775ab3fc4"
format_stamp: "Formatted at 2026-08-12 05:29:20 on dist-test-slave-zpfg"
server_key: "31fddb09758e4a8d57ac4a7517b660b1"
server_key_iv: "758ff69736a5a9cd9e153ef06606bee8"
server_key_version: "encryptionkey@0"
I20260812 05:29:20.264874 13579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:20.283921 13579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:20.284365 13579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:20.284548 13579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:20.284848 13579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 05:29:20.285336 13579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 05:29:20.285393 13579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:20.285425 13579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 05:29:20.285444 13579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:20.291986 13579 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:37667
I20260812 05:29:20.292084 13711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:37667 every 8 connection(s)
I20260812 05:29:20.292567 13579 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/data/info.pb
I20260812 05:29:20.296545 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 13579
I20260812 05:29:20.296698 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/wal/instance
I20260812 05:29:20.296926 24841 external_mini_cluster.cc:1428] Setting key 1bd7f1235fa460a77d86605f3d9c4a9b
I20260812 05:29:20.298804 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.293470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:49759 (local address 127.24.66.126:43075)
0812 05:29:20.293570 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:20.293572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:20.293830 (+ 258us) server_negotiation.cc:407] Connection header received
0812 05:29:20.294428 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:20.294430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:20.294443 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:20.294502 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:20.295242 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.295787 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.296899 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.297187 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.297269 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.298516 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:20.298517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:20.298517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:20.298521 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:20.298537 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:20.298542 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:20.298588 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:20.298710 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 05:29:20.298738 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 05:29:20.299052 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.293438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:43075 (local address 127.24.66.67:49759)
0812 05:29:20.293677 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:20.293715 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:20.294275 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:20.294587 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:20.294591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:20.294600 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:20.295131 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.295136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.295887 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.295889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.296791 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.296795 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.297307 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.297308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.297520 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.298299 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 05:29:20.298300 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:20.298391 (+ 91us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 05:29:20.298422 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:20.298605 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:20.298610 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:20.298612 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:29:20.298704 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 05:29:20.298799 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260812 05:29:20.299584 13712 heartbeater.cc:347] Connected to a master server at 127.24.66.126:43075
I20260812 05:29:20.299700 13712 heartbeater.cc:464] Registering TS with master...
I20260812 05:29:20.299928 13712 heartbeater.cc:511] Master 127.24.66.126:43075 requested a full tablet report, sending...
I20260812 05:29:20.300364 13231 ts_manager.cc:194] Registered new tserver with Master: 34c331b30ad84216b035926775ab3fc4 (127.24.66.67:37667)
I20260812 05:29:20.300823 13231 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:49759
I20260812 05:29:20.309226 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 05:29:20.314172 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.311130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33260 (local address 127.24.66.126:43075)
0812 05:29:20.311187 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:20.311189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:20.311363 (+ 174us) server_negotiation.cc:407] Connection header received
0812 05:29:20.311428 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:20.311429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:20.311441 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:20.311489 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:20.311661 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.312149 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.313003 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.313177 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.313226 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.313961 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:20.313962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:20.313962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:20.313966 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:20.313980 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:20.313984 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:20.314027 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:20.314083 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 05:29:20.314108 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 05:29:20.316227 13231 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 05:29:20.317950 13231 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 05:29:20.324996 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.320458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.67:37667 (local address 127.0.0.1:33340)
0812 05:29:20.320655 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:20.320699 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:20.320770 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:20.321199 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:20.321203 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:20.321205 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:20.321280 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.321283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.322652 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.322653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.323718 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.323720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.324615 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.324616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.324709 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.324724 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 05:29:20.324821 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 05:29:20.324883 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":111,"threads_started":1}
I20260812 05:29:20.325150 13725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.320914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.24.66.65:33299)
0812 05:29:20.321073 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:20.321078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:20.321089 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:29:20.321136 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:20.321138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:20.321150 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:20.321436 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:20.321505 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.322209 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.323911 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.324751 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.324829 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.324915 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:20.325001 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 05:29:20.325054 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260812 05:29:20.325373 13723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.320661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60116 (local address 127.24.66.66:38011)
0812 05:29:20.321319 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:20.321323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:20.321334 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:29:20.321371 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:20.321373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:20.321388 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:20.321608 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:20.321675 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.322751 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.324078 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.324839 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.325121 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.325223 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:20.325247 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 05:29:20.325284 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":114,"threads_started":1}
I20260812 05:29:20.325574 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.320469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:33299 (local address 127.0.0.1:43888)
0812 05:29:20.320873 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:20.320880 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:20.320920 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:20.321294 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:20.321295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:20.321298 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:20.321352 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.321353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.322305 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.322307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.323644 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.323647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.324847 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.324849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.324930 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.324935 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 05:29:20.324989 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 05:29:20.325045 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":123,"threads_started":1}
I20260812 05:29:20.325640 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.320458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:38011 (local address 127.0.0.1:60116)
0812 05:29:20.320700 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:20.320709 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:20.320770 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:20.321486 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:20.321487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:20.321489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:20.321524 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.321525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.322830 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.322831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.323967 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.323970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.324942 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.324944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.325006 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.325009 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:29:20.325043 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 05:29:20.325075 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260812 05:29:20.326507 13507 tablet_service.cc:1467] Processing CreateTablet for tablet 5293e64da22249678f0dcfa1b3432fb0 (DEFAULT_TABLE table=test-table [id=d3c0c568b257458c951781366360f600]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:20.326507 13366 tablet_service.cc:1467] Processing CreateTablet for tablet 5293e64da22249678f0dcfa1b3432fb0 (DEFAULT_TABLE table=test-table [id=d3c0c568b257458c951781366360f600]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:20.326900 13366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5293e64da22249678f0dcfa1b3432fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:20.326900 13507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5293e64da22249678f0dcfa1b3432fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:20.330449 13724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.320742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33340 (local address 127.24.66.67:37667)
0812 05:29:20.321034 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:20.321038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:20.321046 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:29:20.321077 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:20.321079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:20.321091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:20.321181 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:20.321378 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.322565 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.323794 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.324495 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.324580 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.324680 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:20.325637 (+ 957us) server_negotiation.cc:299] Negotiation successful
0812 05:29:20.325971 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":77,"threads_started":1}
I20260812 05:29:20.330636 13727 tablet_bootstrap.cc:492] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc: Bootstrap starting.
I20260812 05:29:20.331226 13643 tablet_service.cc:1467] Processing CreateTablet for tablet 5293e64da22249678f0dcfa1b3432fb0 (DEFAULT_TABLE table=test-table [id=d3c0c568b257458c951781366360f600]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 05:29:20.331440 13727 tablet_bootstrap.cc:654] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:20.331627 13643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5293e64da22249678f0dcfa1b3432fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:20.331787 13727 log.cc:824] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:20.332156 13726 tablet_bootstrap.cc:492] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16: Bootstrap starting.
I20260812 05:29:20.332682 13727 tablet_bootstrap.cc:492] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc: No bootstrap required, opened a new log
I20260812 05:29:20.332772 13727 ts_tablet_manager.cc:1397] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 05:29:20.334973 13726 tablet_bootstrap.cc:654] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:20.335305 13729 tablet_bootstrap.cc:492] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4: Bootstrap starting.
I20260812 05:29:20.335331 13726 log.cc:824] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:20.335474 13727 raft_consensus.cc:348] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564b1f8cb7c548a883cbecb92882de16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38011 } } peers { permanent_uuid: "34c331b30ad84216b035926775ab3fc4" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 37667 } } peers { permanent_uuid: "cadc342dda154e169ee6ebe917eecfdc" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 33299 } }
I20260812 05:29:20.335677 13727 raft_consensus.cc:374] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:20.335721 13727 raft_consensus.cc:729] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cadc342dda154e169ee6ebe917eecfdc, State: Initialized, Role: FOLLOWER
I20260812 05:29:20.335867 13727 consensus_queue.cc:260] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564b1f8cb7c548a883cbecb92882de16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38011 } } peers { permanent_uuid: "34c331b30ad84216b035926775ab3fc4" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 37667 } } peers { permanent_uuid: "cadc342dda154e169ee6ebe917eecfdc" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 33299 } }
I20260812 05:29:20.336172 13726 tablet_bootstrap.cc:492] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16: No bootstrap required, opened a new log
I20260812 05:29:20.336189 13727 ts_tablet_manager.cc:1428] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260812 05:29:20.336236 13441 heartbeater.cc:503] Master 127.24.66.126:43075 was elected leader, sending a full tablet report...
I20260812 05:29:20.336257 13726 ts_tablet_manager.cc:1397] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 05:29:20.336349 13729 tablet_bootstrap.cc:654] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:20.336715 13729 log.cc:824] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:20.337563 13729 tablet_bootstrap.cc:492] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4: No bootstrap required, opened a new log
I20260812 05:29:20.337653 13729 ts_tablet_manager.cc:1397] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 05:29:20.339171 13726 raft_consensus.cc:348] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564b1f8cb7c548a883cbecb92882de16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38011 } } peers { permanent_uuid: "34c331b30ad84216b035926775ab3fc4" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 37667 } } peers { permanent_uuid: "cadc342dda154e169ee6ebe917eecfdc" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 33299 } }
I20260812 05:29:20.339342 13726 raft_consensus.cc:374] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:20.339380 13726 raft_consensus.cc:729] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 564b1f8cb7c548a883cbecb92882de16, State: Initialized, Role: FOLLOWER
I20260812 05:29:20.339496 13726 consensus_queue.cc:260] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564b1f8cb7c548a883cbecb92882de16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38011 } } peers { permanent_uuid: "34c331b30ad84216b035926775ab3fc4" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 37667 } } peers { permanent_uuid: "cadc342dda154e169ee6ebe917eecfdc" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 33299 } }
I20260812 05:29:20.339780 13726 ts_tablet_manager.cc:1428] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 05:29:20.339859 13576 heartbeater.cc:503] Master 127.24.66.126:43075 was elected leader, sending a full tablet report...
I20260812 05:29:20.340271 13729 raft_consensus.cc:348] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564b1f8cb7c548a883cbecb92882de16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38011 } } peers { permanent_uuid: "34c331b30ad84216b035926775ab3fc4" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 37667 } } peers { permanent_uuid: "cadc342dda154e169ee6ebe917eecfdc" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 33299 } }
I20260812 05:29:20.340389 13729 raft_consensus.cc:374] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:20.340441 13729 raft_consensus.cc:729] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c331b30ad84216b035926775ab3fc4, State: Initialized, Role: FOLLOWER
I20260812 05:29:20.340569 13729 consensus_queue.cc:260] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564b1f8cb7c548a883cbecb92882de16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38011 } } peers { permanent_uuid: "34c331b30ad84216b035926775ab3fc4" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 37667 } } peers { permanent_uuid: "cadc342dda154e169ee6ebe917eecfdc" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 33299 } }
I20260812 05:29:20.340824 13729 ts_tablet_manager.cc:1428] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 05:29:20.340901 13712 heartbeater.cc:503] Master 127.24.66.126:43075 was elected leader, sending a full tablet report...
I20260812 05:29:20.357201 13734 raft_consensus.cc:482] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 05:29:20.357332 13734 raft_consensus.cc:504] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564b1f8cb7c548a883cbecb92882de16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38011 } } peers { permanent_uuid: "34c331b30ad84216b035926775ab3fc4" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 37667 } } peers { permanent_uuid: "cadc342dda154e169ee6ebe917eecfdc" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 33299 } }
I20260812 05:29:20.357653 13734 leader_election.cc:288] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 564b1f8cb7c548a883cbecb92882de16 (127.24.66.66:38011), cadc342dda154e169ee6ebe917eecfdc (127.24.66.65:33299)
I20260812 05:29:20.360805 13723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.357905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:37215 (local address 127.24.66.66:38011)
0812 05:29:20.357954 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:20.357956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:20.358130 (+ 174us) server_negotiation.cc:407] Connection header received
0812 05:29:20.358176 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:20.358177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:20.358192 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:20.358246 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:20.358441 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.358904 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.359999 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.360423 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.360484 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.360515 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:20.360667 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 05:29:20.360714 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:29:20.360806 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.357834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:33299 (local address 127.24.66.67:45631)
0812 05:29:20.357906 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:20.357920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:20.357963 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:20.358143 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:20.358144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:20.358146 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:20.358236 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.358239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.359052 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.359053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.359940 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.359942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.360544 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.360545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.360617 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.360619 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 05:29:20.360693 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 05:29:20.360730 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":35}
I20260812 05:29:20.360824 13725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.357937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.66.67:45631 (local address 127.24.66.65:33299)
0812 05:29:20.357981 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:20.357983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:20.357993 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:29:20.358033 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:20.358034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:20.358046 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:20.358121 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 05:29:20.358367 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.358953 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.360067 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.360436 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.360526 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.360533 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:20.360715 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 05:29:20.360757 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:29:20.360824 13735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.357834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.66:38011 (local address 127.24.66.67:37215)
0812 05:29:20.358039 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:20.358051 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:20.358111 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:20.358258 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:20.358259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:20.358260 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 05:29:20.358369 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.358371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.358985 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.358986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.359912 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:20.359915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.360511 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.360512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:20.360598 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.360601 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 05:29:20.360644 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 05:29:20.360709 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":42,"thread_start_us":121,"threads_started":1}
I20260812 05:29:20.361217 13386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5293e64da22249678f0dcfa1b3432fb0" candidate_uuid: "34c331b30ad84216b035926775ab3fc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cadc342dda154e169ee6ebe917eecfdc" is_pre_election: true
I20260812 05:29:20.361361 13527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5293e64da22249678f0dcfa1b3432fb0" candidate_uuid: "34c331b30ad84216b035926775ab3fc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "564b1f8cb7c548a883cbecb92882de16" is_pre_election: true
I20260812 05:29:20.361415 13386 raft_consensus.cc:2454] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34c331b30ad84216b035926775ab3fc4 in term 0.
I20260812 05:29:20.361490 13527 raft_consensus.cc:2454] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34c331b30ad84216b035926775ab3fc4 in term 0.
I20260812 05:29:20.361673 13595 leader_election.cc:302] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c331b30ad84216b035926775ab3fc4, 564b1f8cb7c548a883cbecb92882de16; no voters:
I20260812 05:29:20.361832 13734 raft_consensus.cc:2781] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 05:29:20.361920 13734 raft_consensus.cc:482] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 05:29:20.361953 13734 raft_consensus.cc:3037] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:20.362716 13734 raft_consensus.cc:504] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564b1f8cb7c548a883cbecb92882de16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38011 } } peers { permanent_uuid: "34c331b30ad84216b035926775ab3fc4" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 37667 } } peers { permanent_uuid: "cadc342dda154e169ee6ebe917eecfdc" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 33299 } }
I20260812 05:29:20.362887 13734 leader_election.cc:288] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [CANDIDATE]: Term 1 election: Requested vote from peers 564b1f8cb7c548a883cbecb92882de16 (127.24.66.66:38011), cadc342dda154e169ee6ebe917eecfdc (127.24.66.65:33299)
I20260812 05:29:20.363073 13386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5293e64da22249678f0dcfa1b3432fb0" candidate_uuid: "34c331b30ad84216b035926775ab3fc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cadc342dda154e169ee6ebe917eecfdc"
I20260812 05:29:20.363088 13527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5293e64da22249678f0dcfa1b3432fb0" candidate_uuid: "34c331b30ad84216b035926775ab3fc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "564b1f8cb7c548a883cbecb92882de16"
I20260812 05:29:20.363170 13386 raft_consensus.cc:3037] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:20.363169 13527 raft_consensus.cc:3037] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:20.364189 13527 raft_consensus.cc:2454] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34c331b30ad84216b035926775ab3fc4 in term 1.
I20260812 05:29:20.364184 13386 raft_consensus.cc:2454] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34c331b30ad84216b035926775ab3fc4 in term 1.
I20260812 05:29:20.364423 13595 leader_election.cc:302] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c331b30ad84216b035926775ab3fc4, 564b1f8cb7c548a883cbecb92882de16; no voters:
I20260812 05:29:20.364555 13734 raft_consensus.cc:2781] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:20.364828 13734 raft_consensus.cc:686] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 1 LEADER]: Becoming Leader. State: Replica: 34c331b30ad84216b035926775ab3fc4, State: Running, Role: LEADER
I20260812 05:29:20.364979 13734 consensus_queue.cc:237] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564b1f8cb7c548a883cbecb92882de16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38011 } } peers { permanent_uuid: "34c331b30ad84216b035926775ab3fc4" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 37667 } } peers { permanent_uuid: "cadc342dda154e169ee6ebe917eecfdc" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 33299 } }
I20260812 05:29:20.365983 13231 catalog_manager.cc:5495] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34c331b30ad84216b035926775ab3fc4 (127.24.66.67). New cstate: current_term: 1 leader_uuid: "34c331b30ad84216b035926775ab3fc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564b1f8cb7c548a883cbecb92882de16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34c331b30ad84216b035926775ab3fc4" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cadc342dda154e169ee6ebe917eecfdc" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 33299 } health_report { overall_health: UNKNOWN } } }
I20260812 05:29:20.383360 13724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.381073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33342 (local address 127.24.66.67:37667)
0812 05:29:20.381130 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:20.381132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:20.381147 (+ 15us) server_negotiation.cc:407] Connection header received
0812 05:29:20.381183 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:20.381183 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:20.381202 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:20.381265 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:20.381383 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.381850 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.382571 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:20.382764 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:20.382816 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:20.382997 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:20.383098 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:20.383159 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:20.383191 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 05:29:20.383206 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 05:29:20.385461 13527 raft_consensus.cc:1261] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16 [term 1 FOLLOWER]: Refusing update from remote peer 34c331b30ad84216b035926775ab3fc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:20.385463 13386 raft_consensus.cc:1261] T 5293e64da22249678f0dcfa1b3432fb0 P cadc342dda154e169ee6ebe917eecfdc [term 1 FOLLOWER]: Refusing update from remote peer 34c331b30ad84216b035926775ab3fc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:20.385866 13736 consensus_queue.cc:1035] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "564b1f8cb7c548a883cbecb92882de16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:20.386003 13734 consensus_queue.cc:1035] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cadc342dda154e169ee6ebe917eecfdc" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:20.388639 13740 mvcc.cc:205] Tried to move back new op lower bound from 7317555447335866368 to 7317555447255834624. Current Snapshot: MvccSnapshot[applied={T|T < 7317555447335866368}]
I20260812 05:29:20.388819 13742 mvcc.cc:205] Tried to move back new op lower bound from 7317555447335866368 to 7317555447255834624. Current Snapshot: MvccSnapshot[applied={T|T < 7317555447335866368}]
I20260812 05:29:20.389535 13741 mvcc.cc:205] Tried to move back new op lower bound from 7317555447335866368 to 7317555447255834624. Current Snapshot: MvccSnapshot[applied={T|T < 7317555447335866368}]
I20260812 05:29:20.391103 13231 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:20.391191 13231 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 05:29:20.391966 13231 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01291b952761452ba8ff2b6fa05a0804: Sending DeleteTablet for 3 replicas of tablet 5293e64da22249678f0dcfa1b3432fb0
I20260812 05:29:20.392290 13507 tablet_service.cc:1514] Processing DeleteTablet for tablet 5293e64da22249678f0dcfa1b3432fb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:20 UTC) from {username='slave'} at 127.0.0.1:60116
I20260812 05:29:20.392323 13643 tablet_service.cc:1514] Processing DeleteTablet for tablet 5293e64da22249678f0dcfa1b3432fb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:20 UTC) from {username='slave'} at 127.0.0.1:33340
I20260812 05:29:20.392351 13366 tablet_service.cc:1514] Processing DeleteTablet for tablet 5293e64da22249678f0dcfa1b3432fb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 05:29:20 UTC) from {username='slave'} at 127.0.0.1:43888
I20260812 05:29:20.392573 13750 tablet_replica.cc:330] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16: stopping tablet replica
I20260812 05:29:20.392694 13751 tablet_replica.cc:330] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4: stopping tablet replica
I20260812 05:29:20.392817 13751 raft_consensus.cc:2229] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 05:29:20.392841 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 13299
I20260812 05:29:20.394268 13750 raft_consensus.cc:2229] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 05:29:20.394477 13751 raft_consensus.cc:2258] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:20.394626 13750 raft_consensus.cc:2258] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 05:29:20.395268 13750 ts_tablet_manager.cc:1905] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:20.395525 13751 ts_tablet_manager.cc:1905] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 05:29:20.397179 13750 ts_tablet_manager.cc:1918] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:20.397316 13750 log.cc:1196] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-1/wal/wals/5293e64da22249678f0dcfa1b3432fb0
I20260812 05:29:20.397626 13750 ts_tablet_manager.cc:1939] T 5293e64da22249678f0dcfa1b3432fb0 P 564b1f8cb7c548a883cbecb92882de16: Deleting consensus metadata
I20260812 05:29:20.397826 13751 ts_tablet_manager.cc:1918] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 05:29:20.397949 13751 log.cc:1196] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4: Deleting WAL directory at /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786512301961891-24841-0/minicluster-data/ts-2/wal/wals/5293e64da22249678f0dcfa1b3432fb0
I20260812 05:29:20.398218 13212 catalog_manager.cc:4841] TS 564b1f8cb7c548a883cbecb92882de16 (127.24.66.66:38011): tablet 5293e64da22249678f0dcfa1b3432fb0 (table test-table [id=d3c0c568b257458c951781366360f600]) successfully deleted
I20260812 05:29:20.398308 13751 ts_tablet_manager.cc:1939] T 5293e64da22249678f0dcfa1b3432fb0 P 34c331b30ad84216b035926775ab3fc4: Deleting consensus metadata
I20260812 05:29:20.398825 13215 catalog_manager.cc:4841] TS 34c331b30ad84216b035926775ab3fc4 (127.24.66.67:37667): tablet 5293e64da22249678f0dcfa1b3432fb0 (table test-table [id=d3c0c568b257458c951781366360f600]) successfully deleted
W20260812 05:29:20.406411 13213 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.66.65:33299 (error 108)
I20260812 05:29:20.406615 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 13444
I20260812 05:29:20.407092 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:20.406940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.65:33299 (local address 127.0.0.1:43904)
0812 05:29:20.407004 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:20.407049 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.65:33299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 05:29:20.407372 13213 catalog_manager.cc:4584] TS cadc342dda154e169ee6ebe917eecfdc (127.24.66.65:33299): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5293e64da22249678f0dcfa1b3432fb0: Network error: Client connection negotiation failed: client connection to 127.24.66.65:33299: connect: Connection refused (error 111)
I20260812 05:29:20.414273 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 13579
I20260812 05:29:20.420857 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 13197
2026-08-12T05:29:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (736 ms)
[----------] 6 tests from EncryptionPolicyTest (9131 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:20 dist-test-slave-zpfg krb5kdc[13756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:20 dist-test-slave-zpfg krb5kdc[13756](info): set up 2 sockets
Aug 12 05:29:20 dist-test-slave-zpfg krb5kdc[13756](info): commencing operation
krb5kdc: starting...
W20260812 05:29:21.527076 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:21 dist-test-slave-zpfg krb5kdc[13756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:21.662093 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40849
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:21.748190 13770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:21.748453 13770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:21.748503 13770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:21.749742 13770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:21.749817 13770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:21.749845 13770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:21.749876 13770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:21.749902 13770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:21.751571 13770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40849
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:21.751878 13770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:21.752134 13770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:21.757682 13776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:21.757736 13778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:21.757803 13775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:21.758064 13770 server_base.cc:1034] running on GCE node
I20260812 05:29:21.758476 13770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:21.758754 13770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:21.759922 13770 hybrid_clock.cc:648] HybridClock initialized: now 1786512561759900 us; error 31 us; skew 500 ppm
Aug 12 05:29:21 dist-test-slave-zpfg krb5kdc[13756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512561, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:21.762939 13770 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:21.763514 13770 webserver.cc:460] Webserver started at http://127.0.0.1:33281/ using document root <none> and password file <none>
I20260812 05:29:21.763819 13770 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:21.763875 13770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:21.764030 13770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:21.765612 13770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "bb9242892b694139a08ce333ca28e6de"
format_stamp: "Formatted at 2026-08-12 05:29:21 on dist-test-slave-zpfg"
server_key: "4488f96d6694fe7e7c1fedb08aa59283"
server_key_iv: "da8998733d83e396eac877653bb28cb2"
server_key_version: "encryptionkey@0"
I20260812 05:29:21.766062 13770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "bb9242892b694139a08ce333ca28e6de"
format_stamp: "Formatted at 2026-08-12 05:29:21 on dist-test-slave-zpfg"
server_key: "4488f96d6694fe7e7c1fedb08aa59283"
server_key_iv: "da8998733d83e396eac877653bb28cb2"
server_key_version: "encryptionkey@0"
I20260812 05:29:21.767871 13770 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 05:29:21.768893 13785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:21.769099 13770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:21.769224 13770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "bb9242892b694139a08ce333ca28e6de"
format_stamp: "Formatted at 2026-08-12 05:29:21 on dist-test-slave-zpfg"
server_key: "4488f96d6694fe7e7c1fedb08aa59283"
server_key_iv: "da8998733d83e396eac877653bb28cb2"
server_key_version: "encryptionkey@0"
I20260812 05:29:21.769376 13770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:21.804991 13770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:21.807901 13770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:21.808099 13770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:21.813128 13770 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40849
I20260812 05:29:21.813170 13838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40849 every 8 connection(s)
I20260812 05:29:21.813592 13770 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:21.814652 13839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:21.817080 13839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de: Bootstrap starting.
I20260812 05:29:21.817749 13839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:21.818011 13839 log.cc:824] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:21.818696 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 13770
I20260812 05:29:21.818862 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:21.818935 13839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de: No bootstrap required, opened a new log
I20260812 05:29:21.819108 24841 external_mini_cluster.cc:1428] Setting key 6ea2d3474cbed4545635c79aa08fb8a9
I20260812 05:29:21.821797 13839 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9242892b694139a08ce333ca28e6de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40849 } }
I20260812 05:29:21.821977 13839 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:21.822031 13839 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb9242892b694139a08ce333ca28e6de, State: Initialized, Role: FOLLOWER
I20260812 05:29:21.822196 13839 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9242892b694139a08ce333ca28e6de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40849 } }
I20260812 05:29:21.822455 13839 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:21.822528 13839 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:21.822585 13839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:21.823277 13839 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9242892b694139a08ce333ca28e6de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40849 } }
I20260812 05:29:21.823411 13839 leader_election.cc:302] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb9242892b694139a08ce333ca28e6de; no voters:
I20260812 05:29:21.823621 13839 leader_election.cc:288] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:21.823647 13845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:21.823774 13845 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [term 1 LEADER]: Becoming Leader. State: Replica: bb9242892b694139a08ce333ca28e6de, State: Running, Role: LEADER
I20260812 05:29:21.823868 13845 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9242892b694139a08ce333ca28e6de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40849 } }
I20260812 05:29:21.823961 13839 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:21.824335 13845 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb9242892b694139a08ce333ca28e6de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9242892b694139a08ce333ca28e6de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40849 } } }
I20260812 05:29:21.824350 13846 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb9242892b694139a08ce333ca28e6de. Latest consensus state: current_term: 1 leader_uuid: "bb9242892b694139a08ce333ca28e6de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9242892b694139a08ce333ca28e6de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40849 } } }
I20260812 05:29:21.824442 13845 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:21.824472 13846 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de [sys.catalog]: This master's current role is: LEADER
W20260812 05:29:21.825947 13862 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:21.826018 13862 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:21.826083 13853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:21.826314 13853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:21.828167 13853 catalog_manager.cc:1347] Generated new cluster ID: c1f7a47cb32240b1980d4252281e7e5d
I20260812 05:29:21.828227 13853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:29:21 dist-test-slave-zpfg krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 05:29:21.833653 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:21.819830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55535 (local address 127.0.0.1:40849)
0812 05:29:21.820052 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:21.820060 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:21.820088 (+ 28us) server_negotiation.cc:407] Connection header received
0812 05:29:21.820564 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:21.820571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:21.820624 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:21.820754 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:21.821649 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:21.823309 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:21.824519 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:21.825241 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:21.825344 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:21.828906 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:21.828910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:21.828911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:21.828923 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:21.831198 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:21.831788 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:21.831792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:21.831792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:21.831913 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:21.832238 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:21.832239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:21.832239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:21.832618 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:21.832798 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:21.833302 (+ 504us) server_negotiation.cc:299] Negotiation successful
0812 05:29:21.833415 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":93,"threads_started":1}
I20260812 05:29:21.835153 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:21.844741 13844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:21.836809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55919 (local address 127.0.0.1:40849)
0812 05:29:21.836852 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:21.836854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:21.837020 (+ 166us) server_negotiation.cc:407] Connection header received
0812 05:29:21.837059 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:21.837060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:21.837071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:21.837139 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:21.837813 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:21.838646 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:21.839721 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:21.840064 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:21.840140 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:21.841669 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:21.841671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:21.841671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:21.841679 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:21.843434 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:21.843868 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:21.843869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:21.843870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:21.843932 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:21.844163 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:21.844164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:21.844171 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:21.844312 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:21.844429 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:21.844573 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 05:29:21.844643 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:29:21.846980 13853 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:21.847714 13853 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:21.861653 13853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb9242892b694139a08ce333ca28e6de: Generated new TSK 0
I20260812 05:29:21.861922 13853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:21.868255 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 13770
W20260812 05:29:21.874680 13754 connection.cc:535] client connection to 127.0.0.1:40849 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40849): Connection reset by peer (error 104)
2026-08-12T05:29:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:21 dist-test-slave-zpfg krb5kdc[13872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:21 dist-test-slave-zpfg krb5kdc[13872](info): set up 2 sockets
Aug 12 05:29:21 dist-test-slave-zpfg krb5kdc[13872](info): commencing operation
krb5kdc: starting...
W20260812 05:29:23.031543 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:23 dist-test-slave-zpfg krb5kdc[13872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:23.174036 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42009
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:23.260541 13885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:23.260759 13885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:23.260785 13885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:23.261965 13885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:23.262010 13885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:23.262027 13885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:23.262041 13885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:23.262063 13885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:23.263620 13885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42009
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:23.263887 13885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:23.264158 13885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:23.269675 13891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:23.269646 13893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:23.269802 13885 server_base.cc:1034] running on GCE node
W20260812 05:29:23.269646 13890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:23.270156 13885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:23.270514 13885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:23.271715 13885 hybrid_clock.cc:648] HybridClock initialized: now 1786512563271702 us; error 33 us; skew 500 ppm
Aug 12 05:29:23 dist-test-slave-zpfg krb5kdc[13872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512563, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:23.274588 13885 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:23.275074 13885 webserver.cc:460] Webserver started at http://127.0.0.1:42067/ using document root <none> and password file <none>
I20260812 05:29:23.275345 13885 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:23.275393 13885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:23.275556 13885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:23.276763 13885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "c1031316678447ea93bcbcf864515681"
format_stamp: "Formatted at 2026-08-12 05:29:23 on dist-test-slave-zpfg"
server_key: "0abedc47cd37e019324dfab9dfdb570e"
server_key_iv: "e4806ccdbee7233c01bfb4f2b4895b13"
server_key_version: "encryptionkey@0"
I20260812 05:29:23.277134 13885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "c1031316678447ea93bcbcf864515681"
format_stamp: "Formatted at 2026-08-12 05:29:23 on dist-test-slave-zpfg"
server_key: "0abedc47cd37e019324dfab9dfdb570e"
server_key_iv: "e4806ccdbee7233c01bfb4f2b4895b13"
server_key_version: "encryptionkey@0"
I20260812 05:29:23.278661 13885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:29:23.279682 13900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:23.279910 13885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:23.280010 13885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "c1031316678447ea93bcbcf864515681"
format_stamp: "Formatted at 2026-08-12 05:29:23 on dist-test-slave-zpfg"
server_key: "0abedc47cd37e019324dfab9dfdb570e"
server_key_iv: "e4806ccdbee7233c01bfb4f2b4895b13"
server_key_version: "encryptionkey@0"
I20260812 05:29:23.280090 13885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:23.288125 13885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:23.290897 13885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:23.291052 13885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:23.295585 13885 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42009
I20260812 05:29:23.295653 13955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42009 every 8 connection(s)
I20260812 05:29:23.296083 13885 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:23.297785 13956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:23.300186 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 13885
I20260812 05:29:23.300292 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:23.300525 24841 external_mini_cluster.cc:1428] Setting key 2094f66de71dca331867d093f5f17d24
I20260812 05:29:23.301903 13956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681: Bootstrap starting.
I20260812 05:29:23.302747 13956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:23.303182 13956 log.cc:824] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:23.304076 13956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681: No bootstrap required, opened a new log
I20260812 05:29:23.306666 13956 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1031316678447ea93bcbcf864515681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42009 } }
I20260812 05:29:23.306907 13956 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:23.306960 13956 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1031316678447ea93bcbcf864515681, State: Initialized, Role: FOLLOWER
I20260812 05:29:23.307130 13956 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1031316678447ea93bcbcf864515681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42009 } }
I20260812 05:29:23.307245 13956 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:23.307286 13956 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:23.307340 13956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:23.307672 13959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:23.301147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41359 (local address 127.0.0.1:42009)
0812 05:29:23.301381 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:23.301393 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:23.301432 (+ 39us) server_negotiation.cc:407] Connection header received
0812 05:29:23.302041 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:23.302051 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:23.302121 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:23.302301 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:23.302802 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:23.304334 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:23.305167 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:23.305904 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:23.306061 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:23.306820 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:23.306825 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:23.306825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:23.307058 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:23.307105 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:23.307122 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:23.307220 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:23.307454 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 05:29:23.307487 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":120,"threads_started":1}
I20260812 05:29:23.308174 13956 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1031316678447ea93bcbcf864515681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42009 } }
I20260812 05:29:23.308307 13956 leader_election.cc:302] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1031316678447ea93bcbcf864515681; no voters:
I20260812 05:29:23.308554 13956 leader_election.cc:288] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:23.308604 13961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:23.308852 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:23.308898 13956 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:23.310298 13961 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [term 1 LEADER]: Becoming Leader. State: Replica: c1031316678447ea93bcbcf864515681, State: Running, Role: LEADER
I20260812 05:29:23.310506 13961 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1031316678447ea93bcbcf864515681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42009 } }
I20260812 05:29:23.310938 13977 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1031316678447ea93bcbcf864515681. Latest consensus state: current_term: 1 leader_uuid: "c1031316678447ea93bcbcf864515681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1031316678447ea93bcbcf864515681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42009 } } }
I20260812 05:29:23.311050 13977 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:23.311069 13962 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1031316678447ea93bcbcf864515681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1031316678447ea93bcbcf864515681" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42009 } } }
I20260812 05:29:23.311149 13962 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681 [sys.catalog]: This master's current role is: LEADER
W20260812 05:29:23.311967 13980 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:23.312305 13980 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:23.312372 13981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:23.312551 13981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:23.314644 13959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:23.310465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60919 (local address 127.0.0.1:42009)
0812 05:29:23.310502 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:23.310504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:23.310680 (+ 176us) server_negotiation.cc:407] Connection header received
0812 05:29:23.310753 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:23.310754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:23.310767 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:23.310818 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:23.311369 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:23.312065 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:23.313141 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:23.313412 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:23.313502 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:23.314311 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:23.314313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:23.314313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:23.314318 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:23.314334 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:23.314374 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:23.314439 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:23.314558 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 05:29:23.314571 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 05:29:23.314993 13981 catalog_manager.cc:1347] Generated new cluster ID: 9e35fa30f8174f6d9c15762301cb9345
I20260812 05:29:23.315047 13981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:23.356467 13981 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:23.357087 13981 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:23.360334 13981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1031316678447ea93bcbcf864515681: Generated new TSK 0
I20260812 05:29:23.360569 13981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:23.366969 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 13885
W20260812 05:29:23.374184 13870 connection.cc:535] client connection to 127.0.0.1:42009 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42009): Connection reset by peer (error 104)
2026-08-12T05:29:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:23 dist-test-slave-zpfg krb5kdc[13987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:23 dist-test-slave-zpfg krb5kdc[13987](info): set up 2 sockets
Aug 12 05:29:23 dist-test-slave-zpfg krb5kdc[13987](info): commencing operation
krb5kdc: starting...
W20260812 05:29:24.456034 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:24 dist-test-slave-zpfg krb5kdc[13987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:24.590286 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:24.682065 14000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:24.682291 14000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:24.682317 14000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:24.683501 14000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:24.683549 14000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:24.683563 14000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:24.683576 14000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:24.683588 14000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:24.685021 14000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33521
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:24.685238 14000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:24.685431 14000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:24.690603 14005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:24.690603 14008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:24.690629 14000 server_base.cc:1034] running on GCE node
W20260812 05:29:24.690590 14006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:24.691108 14000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:24.691380 14000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:24.692528 14000 hybrid_clock.cc:648] HybridClock initialized: now 1786512564692507 us; error 38 us; skew 500 ppm
Aug 12 05:29:24 dist-test-slave-zpfg krb5kdc[13987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512564, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:24.701943 14000 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:24.702529 14000 webserver.cc:460] Webserver started at http://127.0.0.1:34765/ using document root <none> and password file <none>
I20260812 05:29:24.702804 14000 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:24.702876 14000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:24.703034 14000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:24.704195 14000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "6e16034694594f84b639df6bb1ce9e04"
format_stamp: "Formatted at 2026-08-12 05:29:24 on dist-test-slave-zpfg"
server_key: "fbefbab139ab4978c5d782a260e30b3a"
server_key_iv: "c98622d80f5a8442b40ecbf85269511d"
server_key_version: "encryptionkey@0"
I20260812 05:29:24.704576 14000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "6e16034694594f84b639df6bb1ce9e04"
format_stamp: "Formatted at 2026-08-12 05:29:24 on dist-test-slave-zpfg"
server_key: "fbefbab139ab4978c5d782a260e30b3a"
server_key_iv: "c98622d80f5a8442b40ecbf85269511d"
server_key_version: "encryptionkey@0"
I20260812 05:29:24.706023 14000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:24.706969 14015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:24.707245 14000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:29:24.707397 14000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "6e16034694594f84b639df6bb1ce9e04"
format_stamp: "Formatted at 2026-08-12 05:29:24 on dist-test-slave-zpfg"
server_key: "fbefbab139ab4978c5d782a260e30b3a"
server_key_iv: "c98622d80f5a8442b40ecbf85269511d"
server_key_version: "encryptionkey@0"
I20260812 05:29:24.707480 14000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:24.711066 14000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:24.713485 14000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:24.713757 14000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:24.718251 14000 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33521
I20260812 05:29:24.718295 14073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33521 every 8 connection(s)
I20260812 05:29:24.718689 14000 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:24.719789 14074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:24.722295 14074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04: Bootstrap starting.
I20260812 05:29:24.722981 14074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:24.723269 14074 log.cc:824] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:24.724037 14074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04: No bootstrap required, opened a new log
I20260812 05:29:24.725864 14074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e16034694594f84b639df6bb1ce9e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
I20260812 05:29:24.726032 14074 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:24.726058 14074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e16034694594f84b639df6bb1ce9e04, State: Initialized, Role: FOLLOWER
I20260812 05:29:24.726219 14074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e16034694594f84b639df6bb1ce9e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
I20260812 05:29:24.726311 14074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:24.726383 14074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:24.726434 14074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:24.726998 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 14000
I20260812 05:29:24.727144 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:24.727216 14074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e16034694594f84b639df6bb1ce9e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
I20260812 05:29:24.727349 14074 leader_election.cc:302] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e16034694594f84b639df6bb1ce9e04; no voters:
I20260812 05:29:24.727408 24841 external_mini_cluster.cc:1428] Setting key d1c5909b13816352effda8884ac92110
I20260812 05:29:24.727559 14074 leader_election.cc:288] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:24.727628 14077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:24.727779 14077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [term 1 LEADER]: Becoming Leader. State: Replica: 6e16034694594f84b639df6bb1ce9e04, State: Running, Role: LEADER
I20260812 05:29:24.727898 14074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:24.727898 14077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e16034694594f84b639df6bb1ce9e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } }
I20260812 05:29:24.728499 14079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e16034694594f84b639df6bb1ce9e04. Latest consensus state: current_term: 1 leader_uuid: "6e16034694594f84b639df6bb1ce9e04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e16034694594f84b639df6bb1ce9e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } } }
I20260812 05:29:24.728591 14079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:24.728567 14078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e16034694594f84b639df6bb1ce9e04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e16034694594f84b639df6bb1ce9e04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33521 } } }
I20260812 05:29:24.728819 14078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:24.729189 14083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:24.729818 14083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:24.731117 14081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:24.728230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45655 (local address 127.0.0.1:33521)
0812 05:29:24.728838 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:24.728850 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:24.728897 (+ 47us) server_negotiation.cc:407] Connection header received
0812 05:29:24.729049 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:24.729059 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:24.729098 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:24.729298 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:24.730177 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:24.730181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:24.730181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:24.730186 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:24.730234 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:24.730262 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:24.730333 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:24.730560 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 05:29:24.730600 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":94,"threads_started":1}
I20260812 05:29:24.732770 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:24.732931 14083 catalog_manager.cc:1347] Generated new cluster ID: 569d7a30d5954f2b9441403efbd75aa2
I20260812 05:29:24.732978 14083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:24.735919 14081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:24.734617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51925 (local address 127.0.0.1:33521)
0812 05:29:24.734658 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:24.734660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:24.734769 (+ 109us) server_negotiation.cc:407] Connection header received
0812 05:29:24.734804 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:24.734805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:24.734811 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:24.734894 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:24.735614 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:24.735615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:24.735616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:24.735621 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:24.735640 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:24.735646 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:24.735725 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:24.735818 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 05:29:24.735834 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:29:24.753093 14083 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:24.753338 14083 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:24.759768 14083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e16034694594f84b639df6bb1ce9e04: Generated new TSK 0
I20260812 05:29:24.760054 14083 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:24.765618 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 14000
2026-08-12T05:29:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:24 dist-test-slave-zpfg krb5kdc[14108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:24 dist-test-slave-zpfg krb5kdc[14108](info): set up 2 sockets
Aug 12 05:29:24 dist-test-slave-zpfg krb5kdc[14108](info): commencing operation
krb5kdc: starting...
W20260812 05:29:25.847601 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:25 dist-test-slave-zpfg krb5kdc[14108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:25.985281 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40417
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:26.074156 14121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:26.074457 14121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:26.074501 14121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:26.075723 14121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:26.075798 14121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:26.075829 14121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:26.075857 14121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:26.075889 14121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:26.077486 14121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:26.077780 14121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:26.078028 14121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:26.083259 14127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:26.083227 14128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:26.083402 14130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:26.083518 14121 server_base.cc:1034] running on GCE node
I20260812 05:29:26.083976 14121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:26.084336 14121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:26.085543 14121 hybrid_clock.cc:648] HybridClock initialized: now 1786512566085523 us; error 33 us; skew 500 ppm
Aug 12 05:29:26 dist-test-slave-zpfg krb5kdc[14108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512566, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:26.088591 14121 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:26.089164 14121 webserver.cc:460] Webserver started at http://127.0.0.1:33715/ using document root <none> and password file <none>
I20260812 05:29:26.089443 14121 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:26.089520 14121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:26.089689 14121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:26.090898 14121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "fea02312d4054d6696708b7239928c56"
format_stamp: "Formatted at 2026-08-12 05:29:26 on dist-test-slave-zpfg"
server_key: "3d95ec241a98709d917e35692ffec2d2"
server_key_iv: "8b610739d5b94a9c74106a7811f4e9c4"
server_key_version: "encryptionkey@0"
I20260812 05:29:26.091275 14121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "fea02312d4054d6696708b7239928c56"
format_stamp: "Formatted at 2026-08-12 05:29:26 on dist-test-slave-zpfg"
server_key: "3d95ec241a98709d917e35692ffec2d2"
server_key_iv: "8b610739d5b94a9c74106a7811f4e9c4"
server_key_version: "encryptionkey@0"
I20260812 05:29:26.092725 14121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:29:26.093555 14137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:26.093771 14121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:26.093917 14121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "fea02312d4054d6696708b7239928c56"
format_stamp: "Formatted at 2026-08-12 05:29:26 on dist-test-slave-zpfg"
server_key: "3d95ec241a98709d917e35692ffec2d2"
server_key_iv: "8b610739d5b94a9c74106a7811f4e9c4"
server_key_version: "encryptionkey@0"
I20260812 05:29:26.094002 14121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:26.107975 14121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:26.110533 14121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:26.110713 14121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:26.115507 14121 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40417
I20260812 05:29:26.115545 14193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40417 every 8 connection(s)
I20260812 05:29:26.115943 14121 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:26.116935 14194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:26.119338 14194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56: Bootstrap starting.
I20260812 05:29:26.119957 14194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:26.120266 14194 log.cc:824] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:26.121021 14194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56: No bootstrap required, opened a new log
I20260812 05:29:26.122066 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 14121
I20260812 05:29:26.122294 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:26.122532 24841 external_mini_cluster.cc:1428] Setting key 17bfc60e30b25ab7bb541f4305d4e8f8
I20260812 05:29:26.123018 14194 raft_consensus.cc:348] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea02312d4054d6696708b7239928c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260812 05:29:26.123230 14194 raft_consensus.cc:374] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:26.123284 14194 raft_consensus.cc:729] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea02312d4054d6696708b7239928c56, State: Initialized, Role: FOLLOWER
I20260812 05:29:26.123416 14194 consensus_queue.cc:260] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea02312d4054d6696708b7239928c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260812 05:29:26.123499 14194 raft_consensus.cc:388] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:26.123528 14194 raft_consensus.cc:482] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:26.123569 14194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:26.124318 14194 raft_consensus.cc:504] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea02312d4054d6696708b7239928c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260812 05:29:26.124475 14194 leader_election.cc:302] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fea02312d4054d6696708b7239928c56; no voters:
I20260812 05:29:26.124656 14194 leader_election.cc:288] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:26.124738 14199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:26.124909 14199 raft_consensus.cc:686] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [term 1 LEADER]: Becoming Leader. State: Replica: fea02312d4054d6696708b7239928c56, State: Running, Role: LEADER
I20260812 05:29:26.124927 14194 sys_catalog.cc:564] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:26.125007 14199 consensus_queue.cc:237] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea02312d4054d6696708b7239928c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260812 05:29:26.125430 14199 sys_catalog.cc:455] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fea02312d4054d6696708b7239928c56. Latest consensus state: current_term: 1 leader_uuid: "fea02312d4054d6696708b7239928c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea02312d4054d6696708b7239928c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } } }
I20260812 05:29:26.125558 14199 sys_catalog.cc:458] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:26.125757 14200 sys_catalog.cc:455] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fea02312d4054d6696708b7239928c56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fea02312d4054d6696708b7239928c56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } } }
I20260812 05:29:26.125885 14200 sys_catalog.cc:458] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:26.126278 14208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:26.126936 14208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:26.128504 14208 catalog_manager.cc:1347] Generated new cluster ID: 37da987bc41e4d489e0af162b6964b58
I20260812 05:29:26.128607 14208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:29:26 dist-test-slave-zpfg krb5kdc[14108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 05:29:26.134799 14198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:26.123175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59683 (local address 127.0.0.1:40417)
0812 05:29:26.123393 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:26.123402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:26.123437 (+ 35us) server_negotiation.cc:407] Connection header received
0812 05:29:26.123565 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:26.123572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:26.123632 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:26.123798 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:26.124452 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:26.125978 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:26.127082 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:26.127715 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:26.127823 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:26.130905 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:26.130909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:26.130909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:26.130922 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:26.133019 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:26.133487 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:26.133490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:26.133491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:26.133619 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:26.133844 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:26.133845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:26.133845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:26.134150 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:26.134306 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:26.134460 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 05:29:26.134599 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":96,"threads_started":1}
I20260812 05:29:26.135880 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:26.145153 14198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:26.137606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41305 (local address 127.0.0.1:40417)
0812 05:29:26.137643 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:26.137644 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:26.137802 (+ 158us) server_negotiation.cc:407] Connection header received
0812 05:29:26.137866 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:26.137866 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:26.137876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:26.137918 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:26.138638 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:26.139362 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:26.140411 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:26.140710 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:26.140769 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:26.142248 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:26.142250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:26.142250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:26.142259 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:26.143952 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:26.144372 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:26.144373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:26.144373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:26.144431 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:26.144630 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:26.144631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:26.144631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:26.144754 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:26.144843 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:26.144977 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 05:29:26.145055 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 05:29:26.162716 14208 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:26.163300 14208 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:26.169749 14208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fea02312d4054d6696708b7239928c56: Generated new TSK 0
I20260812 05:29:26.169992 14208 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:26.183182 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 14121
2026-08-12T05:29:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:26 dist-test-slave-zpfg krb5kdc[14227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:26 dist-test-slave-zpfg krb5kdc[14227](info): set up 2 sockets
Aug 12 05:29:26 dist-test-slave-zpfg krb5kdc[14227](info): commencing operation
krb5kdc: starting...
W20260812 05:29:27.291056 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:27 dist-test-slave-zpfg krb5kdc[14227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:27.425208 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:27.509713 14240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:27.509981 14240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:27.510025 14240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:27.511233 14240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:27.511312 14240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:27.511335 14240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:27.511355 14240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:27.511389 14240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:27.512915 14240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:27.513214 14240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:27.513455 14240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:27.518687 14246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:27.518693 14245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:27.518790 14240 server_base.cc:1034] running on GCE node
W20260812 05:29:27.518692 14248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:27.519193 14240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:27.519529 14240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:27.520699 14240 hybrid_clock.cc:648] HybridClock initialized: now 1786512567520666 us; error 44 us; skew 500 ppm
Aug 12 05:29:27 dist-test-slave-zpfg krb5kdc[14227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512567, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:27.523464 14240 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:27.523900 14240 webserver.cc:460] Webserver started at http://127.0.0.1:46831/ using document root <none> and password file <none>
I20260812 05:29:27.524123 14240 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:27.524166 14240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:27.524257 14240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:27.525346 14240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "26a89b5ceeba4632bacb64eb9ad9bcb2"
format_stamp: "Formatted at 2026-08-12 05:29:27 on dist-test-slave-zpfg"
server_key: "7de7afbb275884fc0f4b54b9e784a324"
server_key_iv: "60fc9058a9b1acc727688674f8207511"
server_key_version: "encryptionkey@0"
I20260812 05:29:27.525671 14240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "26a89b5ceeba4632bacb64eb9ad9bcb2"
format_stamp: "Formatted at 2026-08-12 05:29:27 on dist-test-slave-zpfg"
server_key: "7de7afbb275884fc0f4b54b9e784a324"
server_key_iv: "60fc9058a9b1acc727688674f8207511"
server_key_version: "encryptionkey@0"
I20260812 05:29:27.527078 14240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 05:29:27.527864 14255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:27.528069 14240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:27.528160 14240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "26a89b5ceeba4632bacb64eb9ad9bcb2"
format_stamp: "Formatted at 2026-08-12 05:29:27 on dist-test-slave-zpfg"
server_key: "7de7afbb275884fc0f4b54b9e784a324"
server_key_iv: "60fc9058a9b1acc727688674f8207511"
server_key_version: "encryptionkey@0"
I20260812 05:29:27.528218 14240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:27.537086 14240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:27.539445 14240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:27.539559 14240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:27.543829 14240 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35335
I20260812 05:29:27.543898 14311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35335 every 8 connection(s)
I20260812 05:29:27.544231 14240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:27.545198 14312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:27.547662 14312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2: Bootstrap starting.
I20260812 05:29:27.548308 14312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:27.548619 14312 log.cc:824] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:27.549515 14312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2: No bootstrap required, opened a new log
I20260812 05:29:27.550877 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 14240
I20260812 05:29:27.551031 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:27.551226 24841 external_mini_cluster.cc:1428] Setting key 57cd85910d72aed625617e93cdae890e
I20260812 05:29:27.551631 14312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26a89b5ceeba4632bacb64eb9ad9bcb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35335 } }
I20260812 05:29:27.551875 14312 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:27.551954 14312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a89b5ceeba4632bacb64eb9ad9bcb2, State: Initialized, Role: FOLLOWER
I20260812 05:29:27.552119 14312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26a89b5ceeba4632bacb64eb9ad9bcb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35335 } }
I20260812 05:29:27.552217 14312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:27.552249 14312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:27.552301 14312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:27.553130 14312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26a89b5ceeba4632bacb64eb9ad9bcb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35335 } }
I20260812 05:29:27.553243 14312 leader_election.cc:302] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26a89b5ceeba4632bacb64eb9ad9bcb2; no voters:
I20260812 05:29:27.553433 14312 leader_election.cc:288] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:27.553504 14318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:27.553683 14318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [term 1 LEADER]: Becoming Leader. State: Replica: 26a89b5ceeba4632bacb64eb9ad9bcb2, State: Running, Role: LEADER
I20260812 05:29:27.553705 14312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:27.553781 14318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26a89b5ceeba4632bacb64eb9ad9bcb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35335 } }
I20260812 05:29:27.554188 14318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26a89b5ceeba4632bacb64eb9ad9bcb2. Latest consensus state: current_term: 1 leader_uuid: "26a89b5ceeba4632bacb64eb9ad9bcb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26a89b5ceeba4632bacb64eb9ad9bcb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35335 } } }
I20260812 05:29:27.554198 14319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26a89b5ceeba4632bacb64eb9ad9bcb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26a89b5ceeba4632bacb64eb9ad9bcb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35335 } } }
I20260812 05:29:27.554388 14319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:27.554370 14318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:27.555287 14327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:27.555853 14327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:27.557653 14327 catalog_manager.cc:1347] Generated new cluster ID: 6cc79a11491d4634aca9704f4598607b
I20260812 05:29:27.557756 14327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:27.558080 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:27.551900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37155 (local address 127.0.0.1:35335)
0812 05:29:27.552130 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:27.552140 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:27.552177 (+ 37us) server_negotiation.cc:407] Connection header received
0812 05:29:27.552312 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:27.552319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:27.552381 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:27.552538 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:27.552946 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:27.554408 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:27.555614 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:27.556113 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:27.556242 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:27.557154 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:27.557157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:27.557158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:27.557425 (+ 267us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:27.557466 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:27.557482 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:27.557761 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:27.557797 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 05:29:27.557868 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":98,"threads_started":1}
I20260812 05:29:27.559048 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:27.563508 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:27.560499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47819 (local address 127.0.0.1:35335)
0812 05:29:27.560537 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:27.560540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:27.560665 (+ 125us) server_negotiation.cc:407] Connection header received
0812 05:29:27.560739 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:27.560740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:27.560751 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:27.560795 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:27.560950 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:27.561446 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:27.562159 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:27.562469 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:27.562556 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:27.563250 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:27.563252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:27.563252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:27.563257 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:27.563272 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:27.563276 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:27.563319 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:27.563381 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 05:29:27.563408 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 05:29:27.590494 14327 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:27.591135 14327 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:27.594717 14327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26a89b5ceeba4632bacb64eb9ad9bcb2: Generated new TSK 0
I20260812 05:29:27.594920 14327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:27.597236 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 14240
2026-08-12T05:29:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1413 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:27 dist-test-slave-zpfg krb5kdc[14345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:27 dist-test-slave-zpfg krb5kdc[14345](info): set up 2 sockets
Aug 12 05:29:27 dist-test-slave-zpfg krb5kdc[14345](info): commencing operation
krb5kdc: starting...
W20260812 05:29:28.698578 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:28 dist-test-slave-zpfg krb5kdc[14345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:28.834880 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33213
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:28.924798 14358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:28.925084 14358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:28.925138 14358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:28.926386 14358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:28.926465 14358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:28.926486 14358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:28.926507 14358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:28.926538 14358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:28.928166 14358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:28.928476 14358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:28.928743 14358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:28.934527 14364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:28.934515 14366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:28.934515 14363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:28.934803 14358 server_base.cc:1034] running on GCE node
I20260812 05:29:28.935143 14358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:28.935545 14358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:28.936729 14358 hybrid_clock.cc:648] HybridClock initialized: now 1786512568936683 us; error 62 us; skew 500 ppm
Aug 12 05:29:28 dist-test-slave-zpfg krb5kdc[14345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512568, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:28.939740 14358 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:28.940292 14358 webserver.cc:460] Webserver started at http://127.0.0.1:35889/ using document root <none> and password file <none>
I20260812 05:29:28.940585 14358 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:28.940661 14358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:28.940851 14358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:28.942034 14358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "4a8e37f603eb44e6b3a072e7cfd825ca"
format_stamp: "Formatted at 2026-08-12 05:29:28 on dist-test-slave-zpfg"
server_key: "30ba15cd5919127da24129bd3f4078a2"
server_key_iv: "c82904ee0b91e00195e69af60de30f96"
server_key_version: "encryptionkey@0"
I20260812 05:29:28.942497 14358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "4a8e37f603eb44e6b3a072e7cfd825ca"
format_stamp: "Formatted at 2026-08-12 05:29:28 on dist-test-slave-zpfg"
server_key: "30ba15cd5919127da24129bd3f4078a2"
server_key_iv: "c82904ee0b91e00195e69af60de30f96"
server_key_version: "encryptionkey@0"
I20260812 05:29:28.944006 14358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:28.945151 14375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:28.945360 14358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:28.945487 14358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "4a8e37f603eb44e6b3a072e7cfd825ca"
format_stamp: "Formatted at 2026-08-12 05:29:28 on dist-test-slave-zpfg"
server_key: "30ba15cd5919127da24129bd3f4078a2"
server_key_iv: "c82904ee0b91e00195e69af60de30f96"
server_key_version: "encryptionkey@0"
I20260812 05:29:28.945569 14358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:28.949312 14358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:28.951905 14358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:28.952081 14358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:28.956777 14358 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33213
I20260812 05:29:28.956833 14432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33213 every 8 connection(s)
I20260812 05:29:28.957216 14358 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:28.958259 14433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:28.960788 14433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca: Bootstrap starting.
I20260812 05:29:28.961457 14433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:28.961513 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 14358
I20260812 05:29:28.961649 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:28.961812 14433 log.cc:824] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:28.961861 24841 external_mini_cluster.cc:1428] Setting key 1a903fe773333857886b0397156a5288
I20260812 05:29:28.962790 14433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca: No bootstrap required, opened a new log
I20260812 05:29:28.964555 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:28.962459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37565 (local address 127.0.0.1:33213)
0812 05:29:28.962669 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:28.962677 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:28.962703 (+ 26us) server_negotiation.cc:407] Connection header received
0812 05:29:28.963236 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:28.963244 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:28.963274 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:28.963396 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:28.964086 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:28.964089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:28.964090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:28.964094 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:28.964133 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:28.964143 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:28.964216 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:28.964296 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 05:29:28.964353 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260812 05:29:28.965543 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:28.965687 14433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8e37f603eb44e6b3a072e7cfd825ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33213 } }
I20260812 05:29:28.965858 14433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:28.965891 14433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a8e37f603eb44e6b3a072e7cfd825ca, State: Initialized, Role: FOLLOWER
I20260812 05:29:28.966005 14433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8e37f603eb44e6b3a072e7cfd825ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33213 } }
I20260812 05:29:28.966110 14433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:28.966146 14433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:28.966198 14433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:28.967121 14433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8e37f603eb44e6b3a072e7cfd825ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33213 } }
I20260812 05:29:28.967234 14433 leader_election.cc:302] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a8e37f603eb44e6b3a072e7cfd825ca; no voters:
I20260812 05:29:28.967438 14433 leader_election.cc:288] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:28.967510 14444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:28.967671 14444 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [term 1 LEADER]: Becoming Leader. State: Replica: 4a8e37f603eb44e6b3a072e7cfd825ca, State: Running, Role: LEADER
I20260812 05:29:28.967732 14433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:28.967761 14444 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8e37f603eb44e6b3a072e7cfd825ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33213 } }
I20260812 05:29:28.968384 14445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a8e37f603eb44e6b3a072e7cfd825ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8e37f603eb44e6b3a072e7cfd825ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33213 } } }
I20260812 05:29:28.968483 14445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:28.968791 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:28.967372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47239 (local address 127.0.0.1:33213)
0812 05:29:28.967418 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:28.967420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:28.967564 (+ 144us) server_negotiation.cc:407] Connection header received
0812 05:29:28.967657 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:28.967658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:28.967663 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:28.967693 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:28.968484 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:28.968485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:28.968485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:28.968490 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:28.968505 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:28.968509 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:28.968559 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:28.968698 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 05:29:28.968711 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 05:29:28.968829 14446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a8e37f603eb44e6b3a072e7cfd825ca. Latest consensus state: current_term: 1 leader_uuid: "4a8e37f603eb44e6b3a072e7cfd825ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8e37f603eb44e6b3a072e7cfd825ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33213 } } }
I20260812 05:29:28.968896 14446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca [sys.catalog]: This master's current role is: LEADER
W20260812 05:29:28.969576 14460 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:28.969632 14460 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:28.969697 14453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:28.969910 14453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:28.971626 14453 catalog_manager.cc:1347] Generated new cluster ID: c53c55221d5a47a29cfc7a8a7d52c365
I20260812 05:29:28.971688 14453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:28.984196 14453 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:28.984427 14453 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:28.987246 14453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a8e37f603eb44e6b3a072e7cfd825ca: Generated new TSK 0
I20260812 05:29:28.987452 14453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:28.995370 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 14358
2026-08-12T05:29:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:29 dist-test-slave-zpfg krb5kdc[14466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:29 dist-test-slave-zpfg krb5kdc[14466](info): set up 2 sockets
Aug 12 05:29:29 dist-test-slave-zpfg krb5kdc[14466](info): commencing operation
krb5kdc: starting...
W20260812 05:29:30.087087 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:30 dist-test-slave-zpfg krb5kdc[14466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:30.225216 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34565
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:30.311956 14482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:30.312178 14482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:30.312203 14482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:30.313414 14482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:30.313467 14482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:30.313481 14482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:30.313493 14482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:30.313506 14482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:30.315183 14482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:30.315482 14482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:30.315737 14482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:30.320993 14488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:30.320987 14490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:30.320987 14493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:30.321123 14482 server_base.cc:1034] running on GCE node
I20260812 05:29:30.321556 14482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:30.321851 14482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:30.323009 14482 hybrid_clock.cc:648] HybridClock initialized: now 1786512570322995 us; error 30 us; skew 500 ppm
Aug 12 05:29:30 dist-test-slave-zpfg krb5kdc[14466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512570, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:30.332439 14482 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:30.332975 14482 webserver.cc:460] Webserver started at http://127.0.0.1:41841/ using document root <none> and password file <none>
I20260812 05:29:30.333290 14482 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:30.333374 14482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:30.333539 14482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:30.334793 14482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "c5d9dae2ed87486c8b28db6a60f4775c"
format_stamp: "Formatted at 2026-08-12 05:29:30 on dist-test-slave-zpfg"
server_key: "f269ce9a06f0cb15e7f2e36abc6d0671"
server_key_iv: "be843cdc20af0c1465af601a364157c9"
server_key_version: "encryptionkey@0"
I20260812 05:29:30.335196 14482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "c5d9dae2ed87486c8b28db6a60f4775c"
format_stamp: "Formatted at 2026-08-12 05:29:30 on dist-test-slave-zpfg"
server_key: "f269ce9a06f0cb15e7f2e36abc6d0671"
server_key_iv: "be843cdc20af0c1465af601a364157c9"
server_key_version: "encryptionkey@0"
I20260812 05:29:30.336751 14482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:29:30.337577 14500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:30.337754 14482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:30.337915 14482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "c5d9dae2ed87486c8b28db6a60f4775c"
format_stamp: "Formatted at 2026-08-12 05:29:30 on dist-test-slave-zpfg"
server_key: "f269ce9a06f0cb15e7f2e36abc6d0671"
server_key_iv: "be843cdc20af0c1465af601a364157c9"
server_key_version: "encryptionkey@0"
I20260812 05:29:30.337996 14482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:30.355513 14482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:30.357622 14482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:30.357810 14482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:30.362320 14482 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34565
I20260812 05:29:30.362437 14558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34565 every 8 connection(s)
I20260812 05:29:30.362834 14482 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:30.363734 14559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:30.366210 14559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c: Bootstrap starting.
I20260812 05:29:30.366892 14559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:30.367245 14559 log.cc:824] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:30.368028 14559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c: No bootstrap required, opened a new log
I20260812 05:29:30.369730 14559 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d9dae2ed87486c8b28db6a60f4775c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } }
I20260812 05:29:30.369887 14559 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:30.369912 14559 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d9dae2ed87486c8b28db6a60f4775c, State: Initialized, Role: FOLLOWER
I20260812 05:29:30.370085 14559 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d9dae2ed87486c8b28db6a60f4775c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } }
I20260812 05:29:30.370149 14559 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:30.370172 14559 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:30.370242 14559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:30.370890 14559 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d9dae2ed87486c8b28db6a60f4775c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } }
I20260812 05:29:30.371003 14559 leader_election.cc:302] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5d9dae2ed87486c8b28db6a60f4775c; no voters:
I20260812 05:29:30.371193 14559 leader_election.cc:288] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:30.371232 14562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:30.371472 14559 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:30.371492 14562 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [term 1 LEADER]: Becoming Leader. State: Replica: c5d9dae2ed87486c8b28db6a60f4775c, State: Running, Role: LEADER
I20260812 05:29:30.371606 14562 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d9dae2ed87486c8b28db6a60f4775c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } }
I20260812 05:29:30.371882 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 14482
I20260812 05:29:30.372040 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:30.372258 24841 external_mini_cluster.cc:1428] Setting key d843e4b02cdae13fcdd8c94096472c5b
I20260812 05:29:30.373144 14564 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5d9dae2ed87486c8b28db6a60f4775c. Latest consensus state: current_term: 1 leader_uuid: "c5d9dae2ed87486c8b28db6a60f4775c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d9dae2ed87486c8b28db6a60f4775c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } } }
I20260812 05:29:30.373298 14563 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5d9dae2ed87486c8b28db6a60f4775c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d9dae2ed87486c8b28db6a60f4775c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34565 } } }
I20260812 05:29:30.373561 14564 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:30.373589 14563 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c [sys.catalog]: This master's current role is: LEADER
W20260812 05:29:30.373495 14579 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:30.373701 14579 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:30.374452 14581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:30.374635 14581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:30.376267 14581 catalog_manager.cc:1347] Generated new cluster ID: 8cdd29a02e5f4493ba1b24397cdf3f35
I20260812 05:29:30.376327 14581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:29:30 dist-test-slave-zpfg krb5kdc[14466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 05:29:30.383824 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:30.372761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:56133 (local address 127.0.0.1:34565)
0812 05:29:30.372921 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:30.372930 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:30.372990 (+ 60us) server_negotiation.cc:407] Connection header received
0812 05:29:30.373170 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:30.373178 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:30.373266 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:30.373726 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:30.374410 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:30.375508 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:30.376584 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:30.377013 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:30.377136 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:30.379713 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:30.379717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:30.379718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:30.379732 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:30.382084 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:30.382588 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:30.382591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:30.382592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:30.382702 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:30.382950 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:30.382951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:30.382951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:30.383217 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:30.383423 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:30.383514 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 05:29:30.383625 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":89,"threads_started":1}
I20260812 05:29:30.384923 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:30.392949 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:30.386380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:41571 (local address 127.0.0.1:34565)
0812 05:29:30.386490 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:30.386492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:30.386544 (+ 52us) server_negotiation.cc:407] Connection header received
0812 05:29:30.386607 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:30.386608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:30.386620 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:30.386675 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:30.387230 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:30.387745 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:30.388451 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:30.388696 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:30.388787 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:30.390048 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:30.390050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:30.390051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:30.390058 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:30.391609 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:30.392005 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:30.392006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:30.392007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:30.392064 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:30.392315 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:30.392315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:30.392316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:30.392446 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:30.392559 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:30.392685 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 05:29:30.392753 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 05:29:30.409605 14581 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:30.410404 14581 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:30.414139 14581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5d9dae2ed87486c8b28db6a60f4775c: Generated new TSK 0
I20260812 05:29:30.414387 14581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:30.421260 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 14482
W20260812 05:29:30.428167 14464 connection.cc:535] client connection to 127.0.0.1:34565 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34565): Connection reset by peer (error 104)
2026-08-12T05:29:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:30 dist-test-slave-zpfg krb5kdc[14592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:30 dist-test-slave-zpfg krb5kdc[14592](info): set up 2 sockets
Aug 12 05:29:30 dist-test-slave-zpfg krb5kdc[14592](info): commencing operation
krb5kdc: starting...
W20260812 05:29:31.527748 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:31 dist-test-slave-zpfg krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:31.657768 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41333
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:31.743616 14605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:31.743888 14605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:31.743932 14605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:31.745141 14605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:31.745227 14605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:31.745260 14605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:31.745291 14605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:31.745324 14605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:31.746956 14605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:31.747279 14605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:31.747539 14605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:29:31.752803 14605 server_base.cc:1034] running on GCE node
W20260812 05:29:31.752816 14611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:31.752776 14610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:31.752785 14613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:31.753369 14605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:31.753638 14605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:31.754812 14605 hybrid_clock.cc:648] HybridClock initialized: now 1786512571754799 us; error 32 us; skew 500 ppm
Aug 12 05:29:31 dist-test-slave-zpfg krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512571, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:31.757506 14605 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:31.758031 14605 webserver.cc:460] Webserver started at http://127.0.0.1:36115/ using document root <none> and password file <none>
I20260812 05:29:31.758279 14605 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:31.758392 14605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:31.758548 14605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:31.759692 14605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "0b9f27406caa4036a8b50720b0652f61"
format_stamp: "Formatted at 2026-08-12 05:29:31 on dist-test-slave-zpfg"
server_key: "e13b41f7dedd1fc5684d10522020dfa7"
server_key_iv: "54369446f314a5c017eb7c08c9e624ec"
server_key_version: "encryptionkey@0"
I20260812 05:29:31.760080 14605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "0b9f27406caa4036a8b50720b0652f61"
format_stamp: "Formatted at 2026-08-12 05:29:31 on dist-test-slave-zpfg"
server_key: "e13b41f7dedd1fc5684d10522020dfa7"
server_key_iv: "54369446f314a5c017eb7c08c9e624ec"
server_key_version: "encryptionkey@0"
I20260812 05:29:31.761518 14605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:29:31.762456 14623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:31.762705 14605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:31.762859 14605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "0b9f27406caa4036a8b50720b0652f61"
format_stamp: "Formatted at 2026-08-12 05:29:31 on dist-test-slave-zpfg"
server_key: "e13b41f7dedd1fc5684d10522020dfa7"
server_key_iv: "54369446f314a5c017eb7c08c9e624ec"
server_key_version: "encryptionkey@0"
I20260812 05:29:31.762952 14605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:31.775822 14605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:31.778131 14605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:31.778380 14605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:31.783532 14605 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41333
I20260812 05:29:31.783627 14679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41333 every 8 connection(s)
I20260812 05:29:31.784006 14605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:31.784946 14680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:31.787420 14680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61: Bootstrap starting.
I20260812 05:29:31.788056 14680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:31.788376 14680 log.cc:824] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:31.789156 14680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61: No bootstrap required, opened a new log
I20260812 05:29:31.790912 14680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9f27406caa4036a8b50720b0652f61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } }
I20260812 05:29:31.791059 14680 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:31.791082 14680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b9f27406caa4036a8b50720b0652f61, State: Initialized, Role: FOLLOWER
I20260812 05:29:31.791237 14680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9f27406caa4036a8b50720b0652f61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } }
I20260812 05:29:31.791298 14680 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:31.791319 14680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:31.791380 14680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:31.791934 14680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9f27406caa4036a8b50720b0652f61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } }
I20260812 05:29:31.792045 14680 leader_election.cc:302] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b9f27406caa4036a8b50720b0652f61; no voters:
I20260812 05:29:31.792232 14680 leader_election.cc:288] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:31.792372 14684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:31.792562 14684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [term 1 LEADER]: Becoming Leader. State: Replica: 0b9f27406caa4036a8b50720b0652f61, State: Running, Role: LEADER
I20260812 05:29:31.792629 14680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:31.792718 14684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9f27406caa4036a8b50720b0652f61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } }
I20260812 05:29:31.793114 14685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b9f27406caa4036a8b50720b0652f61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9f27406caa4036a8b50720b0652f61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } } }
I20260812 05:29:31.793236 14685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:31.793109 14686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b9f27406caa4036a8b50720b0652f61. Latest consensus state: current_term: 1 leader_uuid: "0b9f27406caa4036a8b50720b0652f61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9f27406caa4036a8b50720b0652f61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41333 } } }
I20260812 05:29:31.793457 14686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:31.793457 14688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:31.793913 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 14605
I20260812 05:29:31.794028 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:31.794144 14688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:31.794245 24841 external_mini_cluster.cc:1428] Setting key cb116bddf4f735ef42673a780a0af58d
I20260812 05:29:31.796149 14688 catalog_manager.cc:1347] Generated new cluster ID: 0a9ae058942c4c3486e7d9d2f0ae6bc4
I20260812 05:29:31.796207 14688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:31.802546 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:31.794899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:38277 (local address 127.0.0.1:41333)
0812 05:29:31.795463 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:31.795471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:31.795704 (+ 233us) server_negotiation.cc:407] Connection header received
0812 05:29:31.795845 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:31.795853 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:31.795915 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:31.798483 (+ 2568us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:31.798581 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:31.799539 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:31.800391 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:31.800892 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:31.801019 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:31.801770 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:31.801774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:31.801774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:31.801966 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:31.801997 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:31.802008 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:31.802106 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:31.802188 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 05:29:31.802324 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":73,"threads_started":1}
I20260812 05:29:31.803542 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:31.809523 14688 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:31.810231 14688 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:31.810467 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:31.806652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:50917 (local address 127.0.0.1:41333)
0812 05:29:31.806695 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:31.806697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:31.806910 (+ 213us) server_negotiation.cc:407] Connection header received
0812 05:29:31.806971 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:31.806972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:31.806984 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:31.807041 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:31.807193 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:31.808005 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:31.809012 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:31.809364 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:31.809455 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:31.810201 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:31.810203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:31.810203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:31.810207 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:31.810223 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:31.810227 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:31.810282 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:31.810370 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 05:29:31.810385 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":300,"server-negotiator.queue_time_us":20}
I20260812 05:29:31.815596 14688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b9f27406caa4036a8b50720b0652f61: Generated new TSK 0
I20260812 05:29:31.815824 14688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:31.823362 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 14605
W20260812 05:29:31.830313 14590 connection.cc:535] client connection to 127.0.0.1:41333 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41333): Connection reset by peer (error 104)
2026-08-12T05:29:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:31 dist-test-slave-zpfg krb5kdc[14715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:31 dist-test-slave-zpfg krb5kdc[14715](info): set up 2 sockets
Aug 12 05:29:31 dist-test-slave-zpfg krb5kdc[14715](info): commencing operation
krb5kdc: starting...
W20260812 05:29:32.988620 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:33 dist-test-slave-zpfg krb5kdc[14715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:33.121248 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41111
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:33.210238 14729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:33.210568 14729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:33.210614 14729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:33.211918 14729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:33.211999 14729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:33.212023 14729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:33.212045 14729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:33.212075 14729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:33.213692 14729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:33.213992 14729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:33.214251 14729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:33.219885 14738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:33.220141 14729 server_base.cc:1034] running on GCE node
W20260812 05:29:33.219888 14736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:33.219888 14735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:33.220566 14729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:33.220897 14729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:33.222079 14729 hybrid_clock.cc:648] HybridClock initialized: now 1786512573222079 us; error 47 us; skew 500 ppm
Aug 12 05:29:33 dist-test-slave-zpfg krb5kdc[14715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512573, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:33.224921 14729 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:33.225335 14729 webserver.cc:460] Webserver started at http://127.0.0.1:40827/ using document root <none> and password file <none>
I20260812 05:29:33.225550 14729 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:33.225592 14729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:33.225739 14729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:33.226873 14729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "d531f553579d484bbc52800cfb4fc3e5"
format_stamp: "Formatted at 2026-08-12 05:29:33 on dist-test-slave-zpfg"
server_key: "429aad271e2a5502f8de2f06ff8ca3d2"
server_key_iv: "428e543ebce2cc499c2782727cc650b4"
server_key_version: "encryptionkey@0"
I20260812 05:29:33.227319 14729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "d531f553579d484bbc52800cfb4fc3e5"
format_stamp: "Formatted at 2026-08-12 05:29:33 on dist-test-slave-zpfg"
server_key: "429aad271e2a5502f8de2f06ff8ca3d2"
server_key_iv: "428e543ebce2cc499c2782727cc650b4"
server_key_version: "encryptionkey@0"
I20260812 05:29:33.228844 14729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:29:33.229799 14745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:33.230003 14729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:33.230118 14729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "d531f553579d484bbc52800cfb4fc3e5"
format_stamp: "Formatted at 2026-08-12 05:29:33 on dist-test-slave-zpfg"
server_key: "429aad271e2a5502f8de2f06ff8ca3d2"
server_key_iv: "428e543ebce2cc499c2782727cc650b4"
server_key_version: "encryptionkey@0"
I20260812 05:29:33.230297 14729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:33.233794 14729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:33.236280 14729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:33.236454 14729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:33.240860 14729 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41111
I20260812 05:29:33.240902 14802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41111 every 8 connection(s)
I20260812 05:29:33.241276 14729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:33.242331 14803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:33.244802 14803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5: Bootstrap starting.
I20260812 05:29:33.245379 14803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:33.245620 14803 log.cc:824] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:33.246308 14803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5: No bootstrap required, opened a new log
I20260812 05:29:33.247488 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 14729
I20260812 05:29:33.247622 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:33.247819 24841 external_mini_cluster.cc:1428] Setting key 68b0870d34007f28d2f4052cd5a689f8
I20260812 05:29:33.248529 14803 raft_consensus.cc:348] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d531f553579d484bbc52800cfb4fc3e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } }
I20260812 05:29:33.248709 14803 raft_consensus.cc:374] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:33.248736 14803 raft_consensus.cc:729] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d531f553579d484bbc52800cfb4fc3e5, State: Initialized, Role: FOLLOWER
I20260812 05:29:33.248875 14803 consensus_queue.cc:260] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d531f553579d484bbc52800cfb4fc3e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } }
I20260812 05:29:33.248950 14803 raft_consensus.cc:388] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:33.248973 14803 raft_consensus.cc:482] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:33.249038 14803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:33.249954 14803 raft_consensus.cc:504] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d531f553579d484bbc52800cfb4fc3e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } }
I20260812 05:29:33.250084 14803 leader_election.cc:302] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d531f553579d484bbc52800cfb4fc3e5; no voters:
I20260812 05:29:33.250293 14803 leader_election.cc:288] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:33.250314 14809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:33.250458 14809 raft_consensus.cc:686] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [term 1 LEADER]: Becoming Leader. State: Replica: d531f553579d484bbc52800cfb4fc3e5, State: Running, Role: LEADER
I20260812 05:29:33.250602 14809 consensus_queue.cc:237] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d531f553579d484bbc52800cfb4fc3e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } }
I20260812 05:29:33.250612 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:33.248403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:59629 (local address 127.0.0.1:41111)
0812 05:29:33.248748 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:33.248760 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:33.248811 (+ 51us) server_negotiation.cc:407] Connection header received
0812 05:29:33.248958 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:33.248966 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:33.249008 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:33.249182 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:33.250035 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:33.250038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:33.250039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:33.250043 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:33.250095 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:33.250111 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:33.250175 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:33.250249 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 05:29:33.250410 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":118,"threads_started":1}
I20260812 05:29:33.250604 14803 sys_catalog.cc:564] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:33.251060 14810 sys_catalog.cc:455] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d531f553579d484bbc52800cfb4fc3e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d531f553579d484bbc52800cfb4fc3e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } } }
I20260812 05:29:33.251314 14810 sys_catalog.cc:458] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:33.251227 14811 sys_catalog.cc:455] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d531f553579d484bbc52800cfb4fc3e5. Latest consensus state: current_term: 1 leader_uuid: "d531f553579d484bbc52800cfb4fc3e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d531f553579d484bbc52800cfb4fc3e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41111 } } }
I20260812 05:29:33.251545 14811 sys_catalog.cc:458] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:33.251897 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:33.252075 14818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:33.252620 14818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:33.254523 14818 catalog_manager.cc:1347] Generated new cluster ID: 02b04352e91143f5ae73cd33a3501770
I20260812 05:29:33.254585 14818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:33.255961 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:33.254693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:49155 (local address 127.0.0.1:41111)
0812 05:29:33.254730 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:33.254731 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:33.254924 (+ 193us) server_negotiation.cc:407] Connection header received
0812 05:29:33.254990 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:33.254991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:33.254996 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:33.255057 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:33.255712 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:33.255713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:33.255713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:33.255717 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:33.255734 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:33.255739 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:33.255797 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:33.255877 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 05:29:33.255891 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:29:33.272212 14818 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:33.272442 14818 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:33.287358 14818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d531f553579d484bbc52800cfb4fc3e5: Generated new TSK 0
I20260812 05:29:33.287626 14818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:33.293339 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 14729
2026-08-12T05:29:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:33 dist-test-slave-zpfg krb5kdc[14837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:33 dist-test-slave-zpfg krb5kdc[14837](info): set up 2 sockets
Aug 12 05:29:33 dist-test-slave-zpfg krb5kdc[14837](info): commencing operation
krb5kdc: starting...
W20260812 05:29:34.380740 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:34 dist-test-slave-zpfg krb5kdc[14837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:34.516762 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41449
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:34.602701 14851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:34.602974 14851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:34.603014 14851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:34.604233 14851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:34.604318 14851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:34.604353 14851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:34.604382 14851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:34.604413 14851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:34.606010 14851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41449
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:34.606309 14851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:34.606592 14851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 05:29:34.611848 14851 server_base.cc:1034] running on GCE node
W20260812 05:29:34.611814 14856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:34.611814 14859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:34.611805 14857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:34.612419 14851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:34.612735 14851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:34.613909 14851 hybrid_clock.cc:648] HybridClock initialized: now 1786512574613880 us; error 40 us; skew 500 ppm
Aug 12 05:29:34 dist-test-slave-zpfg krb5kdc[14837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512574, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:34.616868 14851 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:34.617419 14851 webserver.cc:460] Webserver started at http://127.0.0.1:42419/ using document root <none> and password file <none>
I20260812 05:29:34.617681 14851 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:34.617753 14851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:34.617905 14851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:34.619110 14851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "05c2da53b6ed473dab1ae58909f2dd80"
format_stamp: "Formatted at 2026-08-12 05:29:34 on dist-test-slave-zpfg"
server_key: "d37a8e76a415f8f4ae86accfa2c624ca"
server_key_iv: "4da03bf4c3dc10fbfe08a8c711904480"
server_key_version: "encryptionkey@0"
I20260812 05:29:34.619508 14851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "05c2da53b6ed473dab1ae58909f2dd80"
format_stamp: "Formatted at 2026-08-12 05:29:34 on dist-test-slave-zpfg"
server_key: "d37a8e76a415f8f4ae86accfa2c624ca"
server_key_iv: "4da03bf4c3dc10fbfe08a8c711904480"
server_key_version: "encryptionkey@0"
I20260812 05:29:34.620977 14851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:29:34.621816 14866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:34.622025 14851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:34.622138 14851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "05c2da53b6ed473dab1ae58909f2dd80"
format_stamp: "Formatted at 2026-08-12 05:29:34 on dist-test-slave-zpfg"
server_key: "d37a8e76a415f8f4ae86accfa2c624ca"
server_key_iv: "4da03bf4c3dc10fbfe08a8c711904480"
server_key_version: "encryptionkey@0"
I20260812 05:29:34.622216 14851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:34.642648 14851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:34.645187 14851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:34.645376 14851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:34.650204 14851 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41449
I20260812 05:29:34.650254 14921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41449 every 8 connection(s)
I20260812 05:29:34.650667 14851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:34.651691 14922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:34.653280 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 14851
I20260812 05:29:34.653419 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:34.653615 24841 external_mini_cluster.cc:1428] Setting key f950a45c8e3fd2de84ac86e588ec0ee0
I20260812 05:29:34.654433 14922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80: Bootstrap starting.
I20260812 05:29:34.655113 14922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:34.655471 14922 log.cc:824] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:34.656222 14922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80: No bootstrap required, opened a new log
I20260812 05:29:34.658512 14922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c2da53b6ed473dab1ae58909f2dd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41449 } }
I20260812 05:29:34.658655 14922 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:34.658679 14922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c2da53b6ed473dab1ae58909f2dd80, State: Initialized, Role: FOLLOWER
I20260812 05:29:34.658911 14922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c2da53b6ed473dab1ae58909f2dd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41449 } }
I20260812 05:29:34.659016 14922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:34.659076 14922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:34.659137 14922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:34.660000 14922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c2da53b6ed473dab1ae58909f2dd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41449 } }
I20260812 05:29:34.660147 14922 leader_election.cc:302] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05c2da53b6ed473dab1ae58909f2dd80; no voters:
I20260812 05:29:34.660334 14922 leader_election.cc:288] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:34.660422 14928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:34.660637 14922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:34.660641 14928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [term 1 LEADER]: Becoming Leader. State: Replica: 05c2da53b6ed473dab1ae58909f2dd80, State: Running, Role: LEADER
I20260812 05:29:34.660853 14928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c2da53b6ed473dab1ae58909f2dd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41449 } }
Aug 12 05:29:34 dist-test-slave-zpfg krb5kdc[14837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 05:29:34.661271 14929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05c2da53b6ed473dab1ae58909f2dd80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c2da53b6ed473dab1ae58909f2dd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41449 } } }
I20260812 05:29:34.661350 14929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:34.661523 14930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05c2da53b6ed473dab1ae58909f2dd80. Latest consensus state: current_term: 1 leader_uuid: "05c2da53b6ed473dab1ae58909f2dd80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c2da53b6ed473dab1ae58909f2dd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41449 } } }
I20260812 05:29:34.661580 14930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:34.662035 14937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:34.662635 14937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:34.664580 14937 catalog_manager.cc:1347] Generated new cluster ID: 9e198394dd5f4161bdc106cf01f2e273
I20260812 05:29:34.664640 14937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:34.666299 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:34.654171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:33609 (local address 127.0.0.1:41449)
0812 05:29:34.654406 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:34.654414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:34.654443 (+ 29us) server_negotiation.cc:407] Connection header received
0812 05:29:34.654914 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:34.654921 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:34.654972 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:34.655096 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:34.655964 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:34.656990 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:34.658153 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:34.658635 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:34.658773 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:34.661906 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:34.661910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:34.661911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:34.661923 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:34.664065 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:34.664577 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:34.664581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:34.664581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:34.664714 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:34.664985 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:34.664986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:34.664986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:34.665324 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:34.665468 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:34.665962 (+ 494us) server_negotiation.cc:299] Negotiation successful
0812 05:29:34.666071 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260812 05:29:34.667399 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:34.675444 14937 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:34.676214 14937 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:34.678861 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:34.668890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:53903 (local address 127.0.0.1:41449)
0812 05:29:34.668931 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:34.668941 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:34.670670 (+ 1729us) server_negotiation.cc:407] Connection header received
0812 05:29:34.670755 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:34.670756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:34.670767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:34.670830 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:34.671533 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:34.672309 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:34.673419 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:34.673740 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:34.673803 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:34.675789 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:34.675790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:34.675791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:34.675798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:34.677506 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:34.677972 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:34.677973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:34.677974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:34.678035 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:34.678277 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:34.678278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:34.678278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:34.678426 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:34.678523 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:34.678693 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 05:29:34.678760 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:29:34.684374 14937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05c2da53b6ed473dab1ae58909f2dd80: Generated new TSK 0
I20260812 05:29:34.684587 14937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:34.689814 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 14851
W20260812 05:29:34.696390 14834 connection.cc:535] client connection to 127.0.0.1:41449 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41449): Connection reset by peer (error 104)
2026-08-12T05:29:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1395 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:34 dist-test-slave-zpfg krb5kdc[14954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:34 dist-test-slave-zpfg krb5kdc[14954](info): set up 2 sockets
Aug 12 05:29:34 dist-test-slave-zpfg krb5kdc[14954](info): commencing operation
krb5kdc: starting...
W20260812 05:29:35.902660 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:35 dist-test-slave-zpfg krb5kdc[14954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:36.037647 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:36.123209 14967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:36.123489 14967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:36.123534 14967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:36.124789 14967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:36.124868 14967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:36.124894 14967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:36.124926 14967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:36.124958 14967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:36.126571 14967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43693
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:36.126890 14967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:36.127142 14967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:36.132658 14972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:36.132695 14973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:36.132700 14975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:36.132863 14967 server_base.cc:1034] running on GCE node
I20260812 05:29:36.133266 14967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:36.133683 14967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:36.134882 14967 hybrid_clock.cc:648] HybridClock initialized: now 1786512576134872 us; error 35 us; skew 500 ppm
Aug 12 05:29:36 dist-test-slave-zpfg krb5kdc[14954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512576, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:36.143226 14967 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:36.143805 14967 webserver.cc:460] Webserver started at http://127.0.0.1:36845/ using document root <none> and password file <none>
I20260812 05:29:36.144080 14967 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:36.144155 14967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:36.144317 14967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:36.145485 14967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "bc71ba40bfd94eccb19e228131a67e52"
format_stamp: "Formatted at 2026-08-12 05:29:36 on dist-test-slave-zpfg"
server_key: "909d2047a37949281a0eac9756310a16"
server_key_iv: "656f96136a057acd21d8dbb3fbcd8671"
server_key_version: "encryptionkey@0"
I20260812 05:29:36.145906 14967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "bc71ba40bfd94eccb19e228131a67e52"
format_stamp: "Formatted at 2026-08-12 05:29:36 on dist-test-slave-zpfg"
server_key: "909d2047a37949281a0eac9756310a16"
server_key_iv: "656f96136a057acd21d8dbb3fbcd8671"
server_key_version: "encryptionkey@0"
I20260812 05:29:36.147600 14967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:29:36.148639 14983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:36.148927 14967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:36.149036 14967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "bc71ba40bfd94eccb19e228131a67e52"
format_stamp: "Formatted at 2026-08-12 05:29:36 on dist-test-slave-zpfg"
server_key: "909d2047a37949281a0eac9756310a16"
server_key_iv: "656f96136a057acd21d8dbb3fbcd8671"
server_key_version: "encryptionkey@0"
I20260812 05:29:36.149101 14967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:36.161872 14967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:36.164409 14967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:36.164534 14967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:36.168872 14967 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43693
I20260812 05:29:36.168917 15039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43693 every 8 connection(s)
I20260812 05:29:36.169329 14967 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:36.170420 15040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:36.172930 15040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52: Bootstrap starting.
I20260812 05:29:36.173554 15040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:36.173817 15040 log.cc:824] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:36.174554 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 14967
I20260812 05:29:36.174682 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:36.174870 15040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52: No bootstrap required, opened a new log
I20260812 05:29:36.174893 24841 external_mini_cluster.cc:1428] Setting key bab70a6d89536302302486bd7c1b203c
I20260812 05:29:36.178079 15040 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc71ba40bfd94eccb19e228131a67e52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43693 } }
I20260812 05:29:36.178277 15040 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:36.178308 15040 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc71ba40bfd94eccb19e228131a67e52, State: Initialized, Role: FOLLOWER
I20260812 05:29:36.178481 15040 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc71ba40bfd94eccb19e228131a67e52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43693 } }
I20260812 05:29:36.178586 15040 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:36.178619 15040 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:36.178663 15040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:36.179340 15040 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc71ba40bfd94eccb19e228131a67e52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43693 } }
I20260812 05:29:36.179432 15040 leader_election.cc:302] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc71ba40bfd94eccb19e228131a67e52; no voters:
I20260812 05:29:36.179692 15045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:36.179665 15040 leader_election.cc:288] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:36.179852 15045 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [term 1 LEADER]: Becoming Leader. State: Replica: bc71ba40bfd94eccb19e228131a67e52, State: Running, Role: LEADER
I20260812 05:29:36.179942 15045 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc71ba40bfd94eccb19e228131a67e52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43693 } }
I20260812 05:29:36.180017 15040 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:36.180311 15046 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc71ba40bfd94eccb19e228131a67e52. Latest consensus state: current_term: 1 leader_uuid: "bc71ba40bfd94eccb19e228131a67e52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc71ba40bfd94eccb19e228131a67e52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43693 } } }
I20260812 05:29:36.180423 15046 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:36.180624 15047 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc71ba40bfd94eccb19e228131a67e52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc71ba40bfd94eccb19e228131a67e52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43693 } } }
I20260812 05:29:36.180687 15047 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:36.180845 15051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:36.181406 15051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:36.181927 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:36.175532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:32827 (local address 127.0.0.1:43693)
0812 05:29:36.175767 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:36.175777 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:36.175811 (+ 34us) server_negotiation.cc:407] Connection header received
0812 05:29:36.176496 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:36.176507 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:36.176588 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:36.176762 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:36.176966 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:36.178441 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:36.179525 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:36.180111 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:36.180586 (+ 475us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:36.181094 (+ 508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:36.181098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:36.181099 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:36.181315 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:36.181353 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:36.181369 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:36.181465 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:36.181611 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 05:29:36.181641 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":119,"threads_started":1}
I20260812 05:29:36.183429 15051 catalog_manager.cc:1347] Generated new cluster ID: 6de77905209245d2a951add1af70a5c0
I20260812 05:29:36.183486 15051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:36.183529 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:36.188045 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:36.184940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:57401 (local address 127.0.0.1:43693)
0812 05:29:36.184979 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:36.184981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:36.185133 (+ 152us) server_negotiation.cc:407] Connection header received
0812 05:29:36.185211 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:36.185212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:36.185222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:36.185271 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:36.185423 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:36.185900 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:36.186739 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:36.186961 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:36.187036 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:36.187762 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:36.187763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:36.187763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:36.187768 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:36.187782 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:36.187786 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:36.187844 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:36.187934 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 05:29:36.187946 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:29:36.217474 15051 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:36.218277 15051 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:36.223922 15051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc71ba40bfd94eccb19e228131a67e52: Generated new TSK 0
I20260812 05:29:36.224157 15051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:36.229808 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 14967
W20260812 05:29:36.236192 14952 connection.cc:535] client connection to 127.0.0.1:43693 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43693): Connection reset by peer (error 104)
2026-08-12T05:29:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:36 dist-test-slave-zpfg krb5kdc[15073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:36 dist-test-slave-zpfg krb5kdc[15073](info): set up 2 sockets
Aug 12 05:29:36 dist-test-slave-zpfg krb5kdc[15073](info): commencing operation
krb5kdc: starting...
W20260812 05:29:37.377897 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:37 dist-test-slave-zpfg krb5kdc[15073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 05:29:37.514384 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36027
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:37.608820 15086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:37.609078 15086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:37.609127 15086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:37.610297 15086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:37.610401 15086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:37.610438 15086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:37.610467 15086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:37.610493 15086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:37.612011 15086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36027
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:37.612314 15086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:37.612563 15086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:37.617909 15095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:37.618077 15086 server_base.cc:1034] running on GCE node
W20260812 05:29:37.618010 15091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:37.617909 15093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:37.618520 15086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:37.618896 15086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:37.620101 15086 hybrid_clock.cc:648] HybridClock initialized: now 1786512577620090 us; error 42 us; skew 500 ppm
Aug 12 05:29:37 dist-test-slave-zpfg krb5kdc[15073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512577, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:37.622998 15086 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 05:29:37.623561 15086 webserver.cc:460] Webserver started at http://127.0.0.1:34321/ using document root <none> and password file <none>
I20260812 05:29:37.623807 15086 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:37.623880 15086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:37.624039 15086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:37.625180 15086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "6910331abb90489d953abe5f3def7b51"
format_stamp: "Formatted at 2026-08-12 05:29:37 on dist-test-slave-zpfg"
server_key: "0e06c48478c159c20fd16146152ed361"
server_key_iv: "4f8a8279b8fa0ae1cd30947886e9b1db"
server_key_version: "encryptionkey@0"
I20260812 05:29:37.625574 15086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "6910331abb90489d953abe5f3def7b51"
format_stamp: "Formatted at 2026-08-12 05:29:37 on dist-test-slave-zpfg"
server_key: "0e06c48478c159c20fd16146152ed361"
server_key_iv: "4f8a8279b8fa0ae1cd30947886e9b1db"
server_key_version: "encryptionkey@0"
I20260812 05:29:37.627146 15086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 05:29:37.628048 15103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:37.628407 15086 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:37.628525 15086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "6910331abb90489d953abe5f3def7b51"
format_stamp: "Formatted at 2026-08-12 05:29:37 on dist-test-slave-zpfg"
server_key: "0e06c48478c159c20fd16146152ed361"
server_key_iv: "4f8a8279b8fa0ae1cd30947886e9b1db"
server_key_version: "encryptionkey@0"
I20260812 05:29:37.628618 15086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:37.632334 15086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:37.634792 15086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:37.634961 15086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:37.639663 15086 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36027
I20260812 05:29:37.639705 15158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36027 every 8 connection(s)
I20260812 05:29:37.640097 15086 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:37.640986 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 15086
I20260812 05:29:37.641127 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:37.641325 24841 external_mini_cluster.cc:1428] Setting key 242ceeae52eb73e825fb4b6c3f04f94b
I20260812 05:29:37.641371 15159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:37.644089 15162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:37.641935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:42943 (local address 127.0.0.1:36027)
0812 05:29:37.642122 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:37.642132 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:37.642176 (+ 44us) server_negotiation.cc:407] Connection header received
0812 05:29:37.642662 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:37.642671 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:37.642708 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:37.642882 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:37.643592 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:37.643596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:37.643596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:37.643600 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:37.643660 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:37.643675 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:37.643754 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:37.643849 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 05:29:37.643897 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":64,"threads_started":1}
I20260812 05:29:37.645241 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:37.645252 15159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51: Bootstrap starting.
I20260812 05:29:37.646061 15159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:37.646498 15159 log.cc:824] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:37.647325 15159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51: No bootstrap required, opened a new log
I20260812 05:29:37.648252 15162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:37.647062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:60973 (local address 127.0.0.1:36027)
0812 05:29:37.647103 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:37.647105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:37.647225 (+ 120us) server_negotiation.cc:407] Connection header received
0812 05:29:37.647306 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:37.647307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:37.647312 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:37.647348 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:37.648035 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:37.648037 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:37.648037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:37.648041 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:37.648055 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:37.648059 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:37.648111 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:37.648164 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 05:29:37.648179 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:29:37.649709 15159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6910331abb90489d953abe5f3def7b51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260812 05:29:37.649868 15159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:37.649894 15159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6910331abb90489d953abe5f3def7b51, State: Initialized, Role: FOLLOWER
I20260812 05:29:37.650049 15159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6910331abb90489d953abe5f3def7b51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260812 05:29:37.650134 15159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:37.650178 15159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:37.650233 15159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:37.650870 15159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6910331abb90489d953abe5f3def7b51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260812 05:29:37.650991 15159 leader_election.cc:302] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6910331abb90489d953abe5f3def7b51; no voters:
I20260812 05:29:37.651176 15159 leader_election.cc:288] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:37.651255 15171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:37.651437 15159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:37.651527 15171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [term 1 LEADER]: Becoming Leader. State: Replica: 6910331abb90489d953abe5f3def7b51, State: Running, Role: LEADER
I20260812 05:29:37.651650 15171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6910331abb90489d953abe5f3def7b51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260812 05:29:37.652061 15173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6910331abb90489d953abe5f3def7b51. Latest consensus state: current_term: 1 leader_uuid: "6910331abb90489d953abe5f3def7b51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6910331abb90489d953abe5f3def7b51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } } }
I20260812 05:29:37.652092 15172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6910331abb90489d953abe5f3def7b51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6910331abb90489d953abe5f3def7b51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } } }
I20260812 05:29:37.652171 15173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:37.652190 15172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:37.652477 15175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:37.653030 15175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:37.659435 15175 catalog_manager.cc:1347] Generated new cluster ID: 56a0dacc02394da586e7862cd352c23b
I20260812 05:29:37.659508 15175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:37.681855 15175 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:37.682090 15175 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:37.689064 15175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6910331abb90489d953abe5f3def7b51: Generated new TSK 0
I20260812 05:29:37.689353 15175 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:37.697031 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 15086
2026-08-12T05:29:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1466 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:37 dist-test-slave-zpfg krb5kdc[15195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:37 dist-test-slave-zpfg krb5kdc[15195](info): set up 2 sockets
Aug 12 05:29:37 dist-test-slave-zpfg krb5kdc[15195](info): commencing operation
krb5kdc: starting...
W20260812 05:29:38.844244 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:38 dist-test-slave-zpfg krb5kdc[15195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:29:38.982523 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:43173
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:43173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:39.069519 15209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:39.069820 15209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:39.069868 15209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:39.071237 15209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:39.071323 15209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:39.071349 15209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:39.071370 15209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:39.071398 15209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:39.073081 15209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:43173
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:43173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:39.073426 15209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:39.073678 15209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:39.078907 15214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:39.078887 15215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:39.078887 15217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:39.079319 15209 server_base.cc:1034] running on GCE node
I20260812 05:29:39.079528 15209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:39.079772 15209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:39.080935 15209 hybrid_clock.cc:648] HybridClock initialized: now 1786512579080919 us; error 31 us; skew 500 ppm
Aug 12 05:29:39 dist-test-slave-zpfg krb5kdc[15195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512579, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:39.084299 15209 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:29:39.084878 15209 webserver.cc:460] Webserver started at http://127.24.66.126:41257/ using document root <none> and password file <none>
I20260812 05:29:39.085163 15209 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:39.085243 15209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:39.085412 15209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:39.086711 15209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "17d1a74067194233b5943190670d4bf0"
format_stamp: "Formatted at 2026-08-12 05:29:39 on dist-test-slave-zpfg"
server_key: "2e4ec2f494afc9a352f75ae5d7826d41"
server_key_iv: "d94daa8e881c2bdcaaab6bb7f0c37b09"
server_key_version: "encryptionkey@0"
I20260812 05:29:39.087136 15209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "17d1a74067194233b5943190670d4bf0"
format_stamp: "Formatted at 2026-08-12 05:29:39 on dist-test-slave-zpfg"
server_key: "2e4ec2f494afc9a352f75ae5d7826d41"
server_key_iv: "d94daa8e881c2bdcaaab6bb7f0c37b09"
server_key_version: "encryptionkey@0"
I20260812 05:29:39.088682 15209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:29:39.089810 15224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:39.090058 15209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:39.090297 15209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "17d1a74067194233b5943190670d4bf0"
format_stamp: "Formatted at 2026-08-12 05:29:39 on dist-test-slave-zpfg"
server_key: "2e4ec2f494afc9a352f75ae5d7826d41"
server_key_iv: "d94daa8e881c2bdcaaab6bb7f0c37b09"
server_key_version: "encryptionkey@0"
I20260812 05:29:39.090448 15209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:39.103609 15209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:39.105981 15209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:39.106276 15209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:39.110800 15209 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:43173
I20260812 05:29:39.110929 15279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:43173 every 8 connection(s)
I20260812 05:29:39.111259 15209 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:39.112140 15281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:39.114533 15281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0: Bootstrap starting.
I20260812 05:29:39.115111 15281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:39.115352 15281 log.cc:824] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:39.116065 15281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0: No bootstrap required, opened a new log
I20260812 05:29:39.117815 15281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d1a74067194233b5943190670d4bf0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43173 } }
I20260812 05:29:39.117964 15281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:39.117986 15281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17d1a74067194233b5943190670d4bf0, State: Initialized, Role: FOLLOWER
I20260812 05:29:39.118089 15281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d1a74067194233b5943190670d4bf0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43173 } }
I20260812 05:29:39.118140 15281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:39.118162 15281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:39.118237 15281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:39.118948 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 15209
I20260812 05:29:39.119109 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:39.119319 24841 external_mini_cluster.cc:1428] Setting key 0464e8debe85e38978dd70cffda8476b
I20260812 05:29:39.119465 15281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d1a74067194233b5943190670d4bf0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43173 } }
I20260812 05:29:39.119596 15281 leader_election.cc:302] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17d1a74067194233b5943190670d4bf0; no voters:
I20260812 05:29:39.119812 15281 leader_election.cc:288] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:39.119999 15287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:39.120088 15281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:39.120330 15287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [term 1 LEADER]: Becoming Leader. State: Replica: 17d1a74067194233b5943190670d4bf0, State: Running, Role: LEADER
I20260812 05:29:39.120452 15287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d1a74067194233b5943190670d4bf0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43173 } }
I20260812 05:29:39.120888 15285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17d1a74067194233b5943190670d4bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d1a74067194233b5943190670d4bf0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43173 } } }
I20260812 05:29:39.121054 15285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:39.121279 15285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17d1a74067194233b5943190670d4bf0. Latest consensus state: current_term: 1 leader_uuid: "17d1a74067194233b5943190670d4bf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d1a74067194233b5943190670d4bf0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 43173 } } }
I20260812 05:29:39.121410 15285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:39.121793 15296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:39.122606 15296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:39.124351 15296 catalog_manager.cc:1347] Generated new cluster ID: f6151f269b594e0baff86a99b6ebe1fb
I20260812 05:29:39.124402 15296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 05:29:39 dist-test-slave-zpfg krb5kdc[15195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:39.130393 15288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:39.119877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60539 (local address 127.24.66.126:43173)
0812 05:29:39.120193 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:39.120204 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:39.120242 (+ 38us) server_negotiation.cc:407] Connection header received
0812 05:29:39.120359 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:39.120366 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:39.120450 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:39.120594 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:39.121606 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:39.122932 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:39.123768 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:39.124257 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:39.124393 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:39.126942 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:39.126946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:39.126946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:39.126964 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:39.128712 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:39.129123 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:39.129128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:39.129129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:39.129237 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:39.129485 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:39.129486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:39.129486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:39.129752 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:39.129944 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:39.130033 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 05:29:39.130141 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":71,"threads_started":1}
I20260812 05:29:39.131408 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:39.137331 15296 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:39.138142 15296 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:39.139421 15288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:39.132953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40413 (local address 127.24.66.126:43173)
0812 05:29:39.132991 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:39.132993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:39.133001 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:29:39.133033 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:39.133034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:39.133042 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:39.133088 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:39.133624 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:39.134134 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:39.134943 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:39.135175 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:39.135253 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:39.136499 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:39.136500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:39.136501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:39.136508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:39.138064 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:39.138541 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:39.138542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:39.138543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:39.138600 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:39.138868 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:39.138869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:39.138869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:39.138984 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:39.139111 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:39.139258 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 05:29:39.139324 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:29:39.148941 15296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17d1a74067194233b5943190670d4bf0: Generated new TSK 0
I20260812 05:29:39.149189 15296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:39.157076 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 15209
W20260812 05:29:39.163441 15193 connection.cc:535] client connection to 127.24.66.126:43173 recv error: Network error: failed to read from TLS socket (remote: 127.24.66.126:43173): Connection reset by peer (error 104)
2026-08-12T05:29:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:39 dist-test-slave-zpfg krb5kdc[15315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:39 dist-test-slave-zpfg krb5kdc[15315](info): set up 2 sockets
Aug 12 05:29:39 dist-test-slave-zpfg krb5kdc[15315](info): commencing operation
krb5kdc: starting...
W20260812 05:29:40.247526 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:40 dist-test-slave-zpfg krb5kdc[15315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:29:40.382490 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:37333
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:38005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:37333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:40.469789 15330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:40.470021 15330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:40.470046 15330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:40.471359 15330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:40.471412 15330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:40.471427 15330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:40.471441 15330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:40.471452 15330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:40.473136 15330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:38005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:37333
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:37333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:40.473429 15330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:40.473706 15330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:40.479250 15335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:40.479317 15336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:40.479364 15338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:40.479612 15330 server_base.cc:1034] running on GCE node
I20260812 05:29:40.479933 15330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:40.480260 15330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:40.481446 15330 hybrid_clock.cc:648] HybridClock initialized: now 1786512580481423 us; error 41 us; skew 500 ppm
Aug 12 05:29:40 dist-test-slave-zpfg krb5kdc[15315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512580, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:40.484290 15330 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:29:40.484786 15330 webserver.cc:460] Webserver started at http://127.24.66.126:43171/ using document root <none> and password file <none>
I20260812 05:29:40.485045 15330 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:40.485095 15330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:40.485229 15330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:40.486584 15330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "d1e5bcfa01cc400cb9b97be33508b062"
format_stamp: "Formatted at 2026-08-12 05:29:40 on dist-test-slave-zpfg"
server_key: "1db85009162e789c311de612424fd1da"
server_key_iv: "ef68f6f3d1e592d85c4bae112f6cd91b"
server_key_version: "encryptionkey@0"
I20260812 05:29:40.486968 15330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "d1e5bcfa01cc400cb9b97be33508b062"
format_stamp: "Formatted at 2026-08-12 05:29:40 on dist-test-slave-zpfg"
server_key: "1db85009162e789c311de612424fd1da"
server_key_iv: "ef68f6f3d1e592d85c4bae112f6cd91b"
server_key_version: "encryptionkey@0"
I20260812 05:29:40.488667 15330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:29:40.489612 15345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:40.489850 15330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:40.489964 15330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "d1e5bcfa01cc400cb9b97be33508b062"
format_stamp: "Formatted at 2026-08-12 05:29:40 on dist-test-slave-zpfg"
server_key: "1db85009162e789c311de612424fd1da"
server_key_iv: "ef68f6f3d1e592d85c4bae112f6cd91b"
server_key_version: "encryptionkey@0"
I20260812 05:29:40.490046 15330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:40.500659 15330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:40.502969 15330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:40.503144 15330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:40.507912 15330 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:37333
I20260812 05:29:40.507956 15400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:37333 every 8 connection(s)
I20260812 05:29:40.508385 15330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:40.508872 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 15330
I20260812 05:29:40.509021 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:40.509375 24841 external_mini_cluster.cc:1428] Setting key 37927a233c0452b61b37cc386865fbf0
I20260812 05:29:40.509584 15401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:40.513324 15401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062: Bootstrap starting.
I20260812 05:29:40.514062 15401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:40.514454 15401 log.cc:824] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:40.515352 15401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062: No bootstrap required, opened a new log
I20260812 05:29:40.516278 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:40.509943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50081 (local address 127.24.66.126:37333)
0812 05:29:40.510207 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:40.510217 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:40.510253 (+ 36us) server_negotiation.cc:407] Connection header received
0812 05:29:40.510901 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:40.510911 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:40.510979 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:40.511144 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:40.511410 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:40.512839 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:40.513757 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:40.514478 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:40.514600 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:40.515483 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:40.515488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:40.515489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:40.515714 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:40.515755 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:40.515770 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:40.515862 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:40.515978 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 05:29:40.516008 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":113,"threads_started":1}
I20260812 05:29:40.517529 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:40.518047 15401 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e5bcfa01cc400cb9b97be33508b062" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37333 } }
I20260812 05:29:40.518242 15401 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:40.518285 15401 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1e5bcfa01cc400cb9b97be33508b062, State: Initialized, Role: FOLLOWER
I20260812 05:29:40.518446 15401 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e5bcfa01cc400cb9b97be33508b062" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37333 } }
I20260812 05:29:40.518532 15401 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:40.518566 15401 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:40.518626 15401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:40.519430 15401 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e5bcfa01cc400cb9b97be33508b062" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37333 } }
I20260812 05:29:40.519558 15401 leader_election.cc:302] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1e5bcfa01cc400cb9b97be33508b062; no voters:
I20260812 05:29:40.519735 15401 leader_election.cc:288] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:40.519797 15412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:40.519972 15412 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [term 1 LEADER]: Becoming Leader. State: Replica: d1e5bcfa01cc400cb9b97be33508b062, State: Running, Role: LEADER
I20260812 05:29:40.520015 15401 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:40.520159 15412 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e5bcfa01cc400cb9b97be33508b062" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37333 } }
I20260812 05:29:40.520643 15413 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1e5bcfa01cc400cb9b97be33508b062" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e5bcfa01cc400cb9b97be33508b062" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37333 } } }
I20260812 05:29:40.520783 15413 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:40.521927 15414 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1e5bcfa01cc400cb9b97be33508b062. Latest consensus state: current_term: 1 leader_uuid: "d1e5bcfa01cc400cb9b97be33508b062" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e5bcfa01cc400cb9b97be33508b062" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37333 } } }
I20260812 05:29:40.522048 15414 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062 [sys.catalog]: This master's current role is: LEADER
W20260812 05:29:40.522185 15429 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:40.522253 15429 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:40.522311 15421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:40.522541 15421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:40.523061 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:40.519203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59559 (local address 127.24.66.126:37333)
0812 05:29:40.519271 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:40.519273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:40.519471 (+ 198us) server_negotiation.cc:407] Connection header received
0812 05:29:40.519540 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:40.519541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:40.519552 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:40.519608 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:40.519807 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:40.520469 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:40.521549 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:40.521868 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:40.521942 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:40.522775 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:40.522776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:40.522777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:40.522782 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:40.522797 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:40.522802 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:40.522835 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:40.522974 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 05:29:40.522987 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:29:40.524216 15421 catalog_manager.cc:1347] Generated new cluster ID: 80230528fe594e8b809178e01c035709
I20260812 05:29:40.524271 15421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:40.536835 15421 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:40.537627 15421 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:40.545342 15421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1e5bcfa01cc400cb9b97be33508b062: Generated new TSK 0
I20260812 05:29:40.545581 15421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:40.554307 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 15330
W20260812 05:29:40.561280 15313 connection.cc:535] client connection to 127.24.66.126:37333 recv error: Network error: failed to read from TLS socket (remote: 127.24.66.126:37333): Connection reset by peer (error 104)
2026-08-12T05:29:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:40 dist-test-slave-zpfg krb5kdc[15434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:40 dist-test-slave-zpfg krb5kdc[15434](info): set up 2 sockets
Aug 12 05:29:40 dist-test-slave-zpfg krb5kdc[15434](info): commencing operation
krb5kdc: starting...
W20260812 05:29:41.651539 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:41 dist-test-slave-zpfg krb5kdc[15434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:29:41.784981 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35463
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:35463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:41.872103 15447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:41.872329 15447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:41.872354 15447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:41.873494 15447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:41.873543 15447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:41.873557 15447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:41.873570 15447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:41.873584 15447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:41.875213 15447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:35463
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:35463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:41.875455 15447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:41.875649 15447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:41.880792 15453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:41.880955 15456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:41.880950 15454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:41.881175 15447 server_base.cc:1034] running on GCE node
I20260812 05:29:41.881464 15447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:41.881731 15447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:41.882926 15447 hybrid_clock.cc:648] HybridClock initialized: now 1786512581882900 us; error 42 us; skew 500 ppm
Aug 12 05:29:41 dist-test-slave-zpfg krb5kdc[15434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512581, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:41.890451 15447 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:29:41.890977 15447 webserver.cc:460] Webserver started at http://127.24.66.126:40481/ using document root <none> and password file <none>
I20260812 05:29:41.891268 15447 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:41.891317 15447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:41.891458 15447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:41.892630 15447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "9b065681c15a4eb781288e63dea26925"
format_stamp: "Formatted at 2026-08-12 05:29:41 on dist-test-slave-zpfg"
server_key: "33b2fe1da80dd7d2248f081ff88cec5f"
server_key_iv: "3de4c96cbc2cc7d69dcbe006d45ed63c"
server_key_version: "encryptionkey@0"
I20260812 05:29:41.893004 15447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "9b065681c15a4eb781288e63dea26925"
format_stamp: "Formatted at 2026-08-12 05:29:41 on dist-test-slave-zpfg"
server_key: "33b2fe1da80dd7d2248f081ff88cec5f"
server_key_iv: "3de4c96cbc2cc7d69dcbe006d45ed63c"
server_key_version: "encryptionkey@0"
I20260812 05:29:41.894526 15447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:41.895406 15463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:41.895609 15447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:41.895707 15447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "9b065681c15a4eb781288e63dea26925"
format_stamp: "Formatted at 2026-08-12 05:29:41 on dist-test-slave-zpfg"
server_key: "33b2fe1da80dd7d2248f081ff88cec5f"
server_key_iv: "3de4c96cbc2cc7d69dcbe006d45ed63c"
server_key_version: "encryptionkey@0"
I20260812 05:29:41.895766 15447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:41.899314 15447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:41.901558 15447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:41.901674 15447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:41.905833 15447 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:35463
I20260812 05:29:41.905907 15519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:35463 every 8 connection(s)
I20260812 05:29:41.906268 15447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:41.907528 15520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:41.910110 15520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925: Bootstrap starting.
I20260812 05:29:41.910852 15520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:41.911111 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 15447
I20260812 05:29:41.911195 15520 log.cc:824] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:41.911340 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:41.911634 24841 external_mini_cluster.cc:1428] Setting key 1998d4378227fdf80ea52235d2a6c675
I20260812 05:29:41.912335 15520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925: No bootstrap required, opened a new log
I20260812 05:29:41.914906 15520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b065681c15a4eb781288e63dea26925" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35463 } }
I20260812 05:29:41.915105 15520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:41.915149 15520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b065681c15a4eb781288e63dea26925, State: Initialized, Role: FOLLOWER
I20260812 05:29:41.915452 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:41.912328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51259 (local address 127.24.66.126:35463)
0812 05:29:41.912535 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:41.912545 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:41.912581 (+ 36us) server_negotiation.cc:407] Connection header received
0812 05:29:41.913884 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:41.913895 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:41.913934 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:41.914131 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:41.914865 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:41.914872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:41.914872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:41.914877 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:41.914927 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:41.914942 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:41.915021 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:41.915068 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 05:29:41.915242 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260812 05:29:41.915287 15520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b065681c15a4eb781288e63dea26925" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35463 } }
I20260812 05:29:41.915699 15520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:41.915773 15520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:41.915820 15520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:41.916574 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:41.916666 15520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b065681c15a4eb781288e63dea26925" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35463 } }
I20260812 05:29:41.916788 15520 leader_election.cc:302] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b065681c15a4eb781288e63dea26925; no voters:
I20260812 05:29:41.917003 15520 leader_election.cc:288] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:41.917029 15526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:41.917173 15526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [term 1 LEADER]: Becoming Leader. State: Replica: 9b065681c15a4eb781288e63dea26925, State: Running, Role: LEADER
I20260812 05:29:41.917299 15520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:41.917320 15526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b065681c15a4eb781288e63dea26925" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35463 } }
I20260812 05:29:41.917691 15526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b065681c15a4eb781288e63dea26925. Latest consensus state: current_term: 1 leader_uuid: "9b065681c15a4eb781288e63dea26925" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b065681c15a4eb781288e63dea26925" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35463 } } }
I20260812 05:29:41.917891 15528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b065681c15a4eb781288e63dea26925" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b065681c15a4eb781288e63dea26925" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 35463 } } }
I20260812 05:29:41.917970 15528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:41.917793 15526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:41.918318 15540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:41.919003 15540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:41.920181 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:41.918784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42739 (local address 127.24.66.126:35463)
0812 05:29:41.918823 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:41.918826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:41.919056 (+ 230us) server_negotiation.cc:407] Connection header received
0812 05:29:41.919162 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:41.919163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:41.919168 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:41.919202 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:41.919926 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:41.919927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:41.919928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:41.919932 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:41.919946 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:41.919950 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:41.920001 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:41.920089 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 05:29:41.920105 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:29:41.921099 15540 catalog_manager.cc:1347] Generated new cluster ID: f21b1f78e3a44dfa944caea043fd3746
I20260812 05:29:41.921155 15540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:41.942015 15540 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:41.942246 15540 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:41.947707 15540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b065681c15a4eb781288e63dea26925: Generated new TSK 0
I20260812 05:29:41.947961 15540 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:41.952979 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 15447
2026-08-12T05:29:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:42 dist-test-slave-zpfg krb5kdc[15554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:42 dist-test-slave-zpfg krb5kdc[15554](info): set up 2 sockets
Aug 12 05:29:42 dist-test-slave-zpfg krb5kdc[15554](info): commencing operation
krb5kdc: starting...
W20260812 05:29:43.065232 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:43 dist-test-slave-zpfg krb5kdc[15554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:29:43.202500 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39275
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:46339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:39275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:43.287793 15568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:43.288070 15568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:43.288113 15568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:43.289345 15568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:43.289424 15568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:43.289448 15568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:43.289469 15568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:43.289497 15568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:43.291087 15568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:46339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:39275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:39275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:43.291411 15568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:43.291661 15568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:43.297489 15574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:43.297511 15573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:43.297539 15568 server_base.cc:1034] running on GCE node
W20260812 05:29:43.297518 15576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:43.298040 15568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:43.298441 15568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:43.299628 15568 hybrid_clock.cc:648] HybridClock initialized: now 1786512583299612 us; error 53 us; skew 500 ppm
Aug 12 05:29:43 dist-test-slave-zpfg krb5kdc[15554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512583, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:43.302796 15568 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:29:43.303385 15568 webserver.cc:460] Webserver started at http://127.24.66.126:33439/ using document root <none> and password file <none>
I20260812 05:29:43.303656 15568 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:43.303714 15568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:43.303854 15568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:43.305006 15568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "a25c51584e054a3584f6c819dbbfb5e0"
format_stamp: "Formatted at 2026-08-12 05:29:43 on dist-test-slave-zpfg"
server_key: "f2ac3bf6f75366245a080cf39582d3d2"
server_key_iv: "5d25735748d4d62933cb215f90c67340"
server_key_version: "encryptionkey@0"
I20260812 05:29:43.305392 15568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "a25c51584e054a3584f6c819dbbfb5e0"
format_stamp: "Formatted at 2026-08-12 05:29:43 on dist-test-slave-zpfg"
server_key: "f2ac3bf6f75366245a080cf39582d3d2"
server_key_iv: "5d25735748d4d62933cb215f90c67340"
server_key_version: "encryptionkey@0"
I20260812 05:29:43.307034 15568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:43.308046 15583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:43.308362 15568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:43.308502 15568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "a25c51584e054a3584f6c819dbbfb5e0"
format_stamp: "Formatted at 2026-08-12 05:29:43 on dist-test-slave-zpfg"
server_key: "f2ac3bf6f75366245a080cf39582d3d2"
server_key_iv: "5d25735748d4d62933cb215f90c67340"
server_key_version: "encryptionkey@0"
I20260812 05:29:43.308593 15568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:43.327069 15568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:43.329885 15568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:43.330093 15568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:43.334875 15568 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:39275
I20260812 05:29:43.334904 15638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:39275 every 8 connection(s)
I20260812 05:29:43.335325 15568 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:43.336400 15639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:43.338958 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 15568
I20260812 05:29:43.339123 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:43.339270 15639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0: Bootstrap starting.
I20260812 05:29:43.339339 24841 external_mini_cluster.cc:1428] Setting key d88611dcdd794c0e702226d9bfa8f9f8
I20260812 05:29:43.340135 15639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:43.340504 15639 log.cc:824] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:43.341351 15639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0: No bootstrap required, opened a new log
I20260812 05:29:43.343550 15639 raft_consensus.cc:348] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25c51584e054a3584f6c819dbbfb5e0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39275 } }
I20260812 05:29:43.343744 15639 raft_consensus.cc:374] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:43.343798 15639 raft_consensus.cc:729] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a25c51584e054a3584f6c819dbbfb5e0, State: Initialized, Role: FOLLOWER
I20260812 05:29:43.343974 15639 consensus_queue.cc:260] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25c51584e054a3584f6c819dbbfb5e0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39275 } }
I20260812 05:29:43.344082 15639 raft_consensus.cc:388] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:43.344133 15639 raft_consensus.cc:482] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:43.344189 15639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:43.345091 15639 raft_consensus.cc:504] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25c51584e054a3584f6c819dbbfb5e0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39275 } }
I20260812 05:29:43.345227 15639 leader_election.cc:302] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a25c51584e054a3584f6c819dbbfb5e0; no voters:
I20260812 05:29:43.345427 15639 leader_election.cc:288] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:43.345469 15645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:43.345698 15645 raft_consensus.cc:686] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [term 1 LEADER]: Becoming Leader. State: Replica: a25c51584e054a3584f6c819dbbfb5e0, State: Running, Role: LEADER
I20260812 05:29:43.345774 15639 sys_catalog.cc:564] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:43.345800 15645 consensus_queue.cc:237] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25c51584e054a3584f6c819dbbfb5e0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39275 } }
I20260812 05:29:43.346218 15646 sys_catalog.cc:455] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a25c51584e054a3584f6c819dbbfb5e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25c51584e054a3584f6c819dbbfb5e0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39275 } } }
I20260812 05:29:43.346307 15646 sys_catalog.cc:458] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:43.346516 15647 sys_catalog.cc:455] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a25c51584e054a3584f6c819dbbfb5e0. Latest consensus state: current_term: 1 leader_uuid: "a25c51584e054a3584f6c819dbbfb5e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25c51584e054a3584f6c819dbbfb5e0" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 39275 } } }
I20260812 05:29:43.346585 15647 sys_catalog.cc:458] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0 [sys.catalog]: This master's current role is: LEADER
Aug 12 05:29:43 dist-test-slave-zpfg krb5kdc[15554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:43.347260 15650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:43.347828 15650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:43.349808 15650 catalog_manager.cc:1347] Generated new cluster ID: 710bd4d0a8954f5b9cd2b9be761c8632
I20260812 05:29:43.349869 15650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:43.352396 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:43.339990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48969 (local address 127.24.66.126:39275)
0812 05:29:43.340208 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:43.340219 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:43.340255 (+ 36us) server_negotiation.cc:407] Connection header received
0812 05:29:43.340835 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:43.340843 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:43.340899 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:43.341059 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:43.341840 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:43.342823 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:43.343633 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:43.344144 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:43.344327 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:43.347851 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:43.347856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:43.347857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:43.347870 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:43.350420 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:43.350873 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:43.350877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:43.350877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:43.351002 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:43.351237 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:43.351238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:43.351239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:43.351594 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:43.351769 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:43.351897 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 05:29:43.352143 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":220,"threads_started":1}
I20260812 05:29:43.353482 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:43.362876 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:43.355022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49909 (local address 127.24.66.126:39275)
0812 05:29:43.355058 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:43.355060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:43.355262 (+ 202us) server_negotiation.cc:407] Connection header received
0812 05:29:43.355296 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:43.355297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:43.355313 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:43.355393 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:43.356001 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:43.356886 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:43.357877 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:43.358274 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:43.358417 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:43.359828 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:43.359830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:43.359830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:43.359838 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:43.361589 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:43.361970 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:43.361971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:43.361971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:43.362049 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:43.362258 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:43.362259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:43.362259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:43.362440 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:43.362538 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:43.362676 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 05:29:43.362767 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 05:29:43.366292 15650 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:43.367173 15650 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:43.372278 15650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a25c51584e054a3584f6c819dbbfb5e0: Generated new TSK 0
I20260812 05:29:43.372526 15650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:43.376022 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 15568
2026-08-12T05:29:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:43 dist-test-slave-zpfg krb5kdc[15672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:43 dist-test-slave-zpfg krb5kdc[15672](info): set up 2 sockets
Aug 12 05:29:43 dist-test-slave-zpfg krb5kdc[15672](info): commencing operation
krb5kdc: starting...
W20260812 05:29:44.572496 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:44 dist-test-slave-zpfg krb5kdc[15672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:29:44.705760 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:37017
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:36181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:37017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:44.794462 15685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:44.794736 15685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:44.794775 15685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:44.795991 15685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:44.796067 15685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:44.796113 15685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:44.796146 15685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:44.796167 15685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:44.797748 15685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:36181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:37017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:37017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:44.798058 15685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:44.798307 15685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:44.803665 15693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:44.803805 15692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:44.803665 15695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:44.803949 15685 server_base.cc:1034] running on GCE node
I20260812 05:29:44.804219 15685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:44.804559 15685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:44.805723 15685 hybrid_clock.cc:648] HybridClock initialized: now 1786512584805707 us; error 28 us; skew 500 ppm
Aug 12 05:29:44 dist-test-slave-zpfg krb5kdc[15672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512584, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:44.808723 15685 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:29:44.809249 15685 webserver.cc:460] Webserver started at http://127.24.66.126:35779/ using document root <none> and password file <none>
I20260812 05:29:44.809505 15685 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:44.809584 15685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:44.809748 15685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:44.811115 15685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "7f66c1bfbf0b4900aceb8868918a6739"
format_stamp: "Formatted at 2026-08-12 05:29:44 on dist-test-slave-zpfg"
server_key: "212d49f3e09946cfe1454604518777e7"
server_key_iv: "9ee5e3b366eba93843d8a5b64e98a26c"
server_key_version: "encryptionkey@0"
I20260812 05:29:44.811581 15685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "7f66c1bfbf0b4900aceb8868918a6739"
format_stamp: "Formatted at 2026-08-12 05:29:44 on dist-test-slave-zpfg"
server_key: "212d49f3e09946cfe1454604518777e7"
server_key_iv: "9ee5e3b366eba93843d8a5b64e98a26c"
server_key_version: "encryptionkey@0"
I20260812 05:29:44.813138 15685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:29:44.814035 15702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:44.814221 15685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:44.814417 15685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "7f66c1bfbf0b4900aceb8868918a6739"
format_stamp: "Formatted at 2026-08-12 05:29:44 on dist-test-slave-zpfg"
server_key: "212d49f3e09946cfe1454604518777e7"
server_key_iv: "9ee5e3b366eba93843d8a5b64e98a26c"
server_key_version: "encryptionkey@0"
I20260812 05:29:44.814505 15685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:44.833918 15685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:44.836354 15685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:44.836575 15685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:44.841631 15685 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:37017
I20260812 05:29:44.841692 15757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:37017 every 8 connection(s)
I20260812 05:29:44.842118 15685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:44.843297 15758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:44.846087 15758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739: Bootstrap starting.
I20260812 05:29:44.846829 15758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:44.847178 15758 log.cc:824] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:44.847993 15758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739: No bootstrap required, opened a new log
I20260812 05:29:44.849771 15758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f66c1bfbf0b4900aceb8868918a6739" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37017 } }
I20260812 05:29:44.849912 15758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:44.849936 15758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f66c1bfbf0b4900aceb8868918a6739, State: Initialized, Role: FOLLOWER
I20260812 05:29:44.850134 15758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f66c1bfbf0b4900aceb8868918a6739" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37017 } }
I20260812 05:29:44.850224 15758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:44.850270 15758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:44.850319 15758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:44.851289 15758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f66c1bfbf0b4900aceb8868918a6739" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37017 } }
I20260812 05:29:44.851382 15758 leader_election.cc:302] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f66c1bfbf0b4900aceb8868918a6739; no voters:
I20260812 05:29:44.851576 15758 leader_election.cc:288] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:44.851651 15761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:44.851878 15761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [term 1 LEADER]: Becoming Leader. State: Replica: 7f66c1bfbf0b4900aceb8868918a6739, State: Running, Role: LEADER
I20260812 05:29:44.851958 15758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:44.851971 15761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f66c1bfbf0b4900aceb8868918a6739" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37017 } }
I20260812 05:29:44.852201 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 15685
I20260812 05:29:44.852329 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:44.852550 24841 external_mini_cluster.cc:1428] Setting key 0b0763d9cab36ce5cb6f6c2e7bad5dcd
I20260812 05:29:44.853411 15764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f66c1bfbf0b4900aceb8868918a6739. Latest consensus state: current_term: 1 leader_uuid: "7f66c1bfbf0b4900aceb8868918a6739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f66c1bfbf0b4900aceb8868918a6739" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37017 } } }
I20260812 05:29:44.853554 15764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:44.853576 15762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f66c1bfbf0b4900aceb8868918a6739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f66c1bfbf0b4900aceb8868918a6739" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 37017 } } }
I20260812 05:29:44.853646 15762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739 [sys.catalog]: This master's current role is: LEADER
W20260812 05:29:44.853983 15780 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:44.854038 15780 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:44.854097 15782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:44.854274 15782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:44.855993 15782 catalog_manager.cc:1347] Generated new cluster ID: be590d5bb60443bd87aeac5254630db0
I20260812 05:29:44.856048 15782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:44.859289 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:44.853104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57331 (local address 127.24.66.126:37017)
0812 05:29:44.853399 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:44.853410 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:44.853571 (+ 161us) server_negotiation.cc:407] Connection header received
0812 05:29:44.853845 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:44.853853 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:44.853930 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:44.854024 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:44.854493 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:44.855722 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:44.856915 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:44.857600 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:44.857742 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:44.858537 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:44.858541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:44.858541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:44.858772 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:44.858814 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:44.858829 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:44.858905 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:44.858970 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 05:29:44.859047 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":77,"threads_started":1}
I20260812 05:29:44.860350 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:44.865669 15778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:44.862115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43933 (local address 127.24.66.126:37017)
0812 05:29:44.862156 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:44.862158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:44.862217 (+ 59us) server_negotiation.cc:407] Connection header received
0812 05:29:44.862278 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:44.862278 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:44.862290 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:44.862373 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:44.862503 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:44.863299 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:44.864288 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:44.864602 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:44.864679 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:44.865403 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:44.865404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:44.865405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:44.865409 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:44.865426 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:44.865430 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:44.865491 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:44.865545 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 05:29:44.865574 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:29:44.890183 15782 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:44.890796 15782 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:44.900482 15782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f66c1bfbf0b4900aceb8868918a6739: Generated new TSK 0
I20260812 05:29:44.900755 15782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:44.903903 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 15685
2026-08-12T05:29:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1526 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:44 dist-test-slave-zpfg krb5kdc[15795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:44 dist-test-slave-zpfg krb5kdc[15795](info): set up 2 sockets
Aug 12 05:29:44 dist-test-slave-zpfg krb5kdc[15795](info): commencing operation
krb5kdc: starting...
W20260812 05:29:45.994786 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:46 dist-test-slave-zpfg krb5kdc[15795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:29:46.133908 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:36717
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:43571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:36717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:46.220911 15810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:46.221148 15810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:46.221175 15810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:46.222429 15810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:46.222476 15810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:46.222491 15810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:46.222503 15810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:46.222517 15810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:46.224066 15810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:43571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:36717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:36717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:46.224340 15810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:46.224584 15810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:46.229842 15815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:46.229826 15818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:46.230016 15810 server_base.cc:1034] running on GCE node
W20260812 05:29:46.229919 15816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:46.230494 15810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:46.230782 15810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:46.231949 15810 hybrid_clock.cc:648] HybridClock initialized: now 1786512586231896 us; error 67 us; skew 500 ppm
Aug 12 05:29:46 dist-test-slave-zpfg krb5kdc[15795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512586, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:46.234939 15810 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:29:46.235450 15810 webserver.cc:460] Webserver started at http://127.24.66.126:37523/ using document root <none> and password file <none>
I20260812 05:29:46.235649 15810 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:46.235690 15810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:46.235787 15810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:46.236848 15810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "b62b9a0f5f5743deb2e514f47d72f421"
format_stamp: "Formatted at 2026-08-12 05:29:46 on dist-test-slave-zpfg"
server_key: "e6306d8d152ac4fabb51f397af0637c3"
server_key_iv: "678effae308e02b98fe7438a4bee7c14"
server_key_version: "encryptionkey@0"
I20260812 05:29:46.237161 15810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "b62b9a0f5f5743deb2e514f47d72f421"
format_stamp: "Formatted at 2026-08-12 05:29:46 on dist-test-slave-zpfg"
server_key: "e6306d8d152ac4fabb51f397af0637c3"
server_key_iv: "678effae308e02b98fe7438a4bee7c14"
server_key_version: "encryptionkey@0"
I20260812 05:29:46.238548 15810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:29:46.239346 15826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:46.239532 15810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:46.239645 15810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "b62b9a0f5f5743deb2e514f47d72f421"
format_stamp: "Formatted at 2026-08-12 05:29:46 on dist-test-slave-zpfg"
server_key: "e6306d8d152ac4fabb51f397af0637c3"
server_key_iv: "678effae308e02b98fe7438a4bee7c14"
server_key_version: "encryptionkey@0"
I20260812 05:29:46.239725 15810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:46.243417 15810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:46.246079 15810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:46.246248 15810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:46.250689 15810 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:36717
I20260812 05:29:46.250725 15886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:36717 every 8 connection(s)
I20260812 05:29:46.251093 15810 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:46.252105 15888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:46.254551 15888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421: Bootstrap starting.
I20260812 05:29:46.255157 15888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:46.255431 15888 log.cc:824] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:46.256215 15888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421: No bootstrap required, opened a new log
I20260812 05:29:46.257875 15888 raft_consensus.cc:348] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62b9a0f5f5743deb2e514f47d72f421" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36717 } }
I20260812 05:29:46.258044 15888 raft_consensus.cc:374] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 05:29:46.258067 15888 raft_consensus.cc:729] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b62b9a0f5f5743deb2e514f47d72f421, State: Initialized, Role: FOLLOWER
I20260812 05:29:46.258175 15888 consensus_queue.cc:260] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62b9a0f5f5743deb2e514f47d72f421" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36717 } }
I20260812 05:29:46.258227 15888 raft_consensus.cc:388] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 05:29:46.258248 15888 raft_consensus.cc:482] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 05:29:46.258318 15888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:46.259282 15888 raft_consensus.cc:504] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62b9a0f5f5743deb2e514f47d72f421" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36717 } }
I20260812 05:29:46.259424 15888 leader_election.cc:302] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b62b9a0f5f5743deb2e514f47d72f421; no voters:
I20260812 05:29:46.259622 15888 leader_election.cc:288] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 05:29:46.259697 15891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:46.259871 15891 raft_consensus.cc:686] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [term 1 LEADER]: Becoming Leader. State: Replica: b62b9a0f5f5743deb2e514f47d72f421, State: Running, Role: LEADER
I20260812 05:29:46.259902 15888 sys_catalog.cc:564] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:46.260002 15891 consensus_queue.cc:237] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62b9a0f5f5743deb2e514f47d72f421" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36717 } }
I20260812 05:29:46.260494 15892 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b62b9a0f5f5743deb2e514f47d72f421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62b9a0f5f5743deb2e514f47d72f421" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36717 } } }
I20260812 05:29:46.260573 15892 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:46.260620 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 15810
I20260812 05:29:46.260746 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:46.260973 24841 external_mini_cluster.cc:1428] Setting key cc1a47a73f00eed0917bd9bd852c1de9
I20260812 05:29:46.261008 15893 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b62b9a0f5f5743deb2e514f47d72f421. Latest consensus state: current_term: 1 leader_uuid: "b62b9a0f5f5743deb2e514f47d72f421" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62b9a0f5f5743deb2e514f47d72f421" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 36717 } } }
I20260812 05:29:46.261114 15893 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:46.261519 15900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:46.262049 15900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:46.263849 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:46.261632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36561 (local address 127.24.66.126:36717)
0812 05:29:46.261794 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:46.261804 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:46.261839 (+ 35us) server_negotiation.cc:407] Connection header received
0812 05:29:46.261967 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:46.261974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:46.262005 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:46.262688 (+ 683us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:46.263227 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:46.263231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:46.263231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:46.263235 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:46.263272 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:46.263287 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:46.263358 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:46.263472 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 05:29:46.263663 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":93,"threads_started":1}
I20260812 05:29:46.264349 15900 catalog_manager.cc:1347] Generated new cluster ID: c93d0304f1e8453b8d371298c2e061e6
I20260812 05:29:46.264420 15900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:46.269804 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:46.270041 15900 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:46.270193 15900 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:46.272665 15908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:46.271525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39151 (local address 127.24.66.126:36717)
0812 05:29:46.271562 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:46.271564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:46.271687 (+ 123us) server_negotiation.cc:407] Connection header received
0812 05:29:46.271756 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:46.271756 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:46.271762 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:46.271803 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:46.272442 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:46.272443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:46.272444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 05:29:46.272447 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:46.272462 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 05:29:46.272466 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:46.272519 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:46.272584 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 05:29:46.272598 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:29:46.286396 15900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b62b9a0f5f5743deb2e514f47d72f421: Generated new TSK 0
I20260812 05:29:46.286677 15900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 05:29:46.291180 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 15810
2026-08-12T05:29:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1384 ms)
[----------] 18 tests from AuthTokenIssuingTest (25868 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:46 dist-test-slave-zpfg krb5kdc[15919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:46 dist-test-slave-zpfg krb5kdc[15919](info): set up 2 sockets
Aug 12 05:29:46 dist-test-slave-zpfg krb5kdc[15919](info): commencing operation
krb5kdc: starting...
W20260812 05:29:47.475069 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:47 dist-test-slave-zpfg krb5kdc[15919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:29:47.615418 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:34087
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:34087,127.24.66.125:35655,127.24.66.124:34091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:47.700403 15933 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:29:47.700695 15933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:47.700728 15933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:47.700811 15933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:47.702039 15933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:47.702114 15933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:47.702138 15933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:47.702159 15933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:47.702193 15933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:47.703823 15933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:34087,127.24.66.125:35655,127.24.66.124:34091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:47.704136 15933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:47.704397 15933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:47.709589 15941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:47.709623 15938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:47.709619 15939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:47.709780 15933 server_base.cc:1034] running on GCE node
I20260812 05:29:47.710098 15933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:47.710469 15933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:47.711675 15933 hybrid_clock.cc:648] HybridClock initialized: now 1786512587711660 us; error 30 us; skew 500 ppm
Aug 12 05:29:47 dist-test-slave-zpfg krb5kdc[15919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:47.719416 15933 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:29:47.719892 15933 webserver.cc:460] Webserver started at http://127.24.66.126:34915/ using document root <none> and password file <none>
I20260812 05:29:47.720149 15933 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:47.720222 15933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:47.720381 15933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:47.721467 15933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "97056a4d6a9d4984935541eeefac4699"
format_stamp: "Formatted at 2026-08-12 05:29:47 on dist-test-slave-zpfg"
server_key: "6a2cdb3437b4791875df14ad5f94bf4c"
server_key_iv: "1e9832522ee698b2cbdaa6903b8b31cd"
server_key_version: "encryptionkey@0"
I20260812 05:29:47.721832 15933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "97056a4d6a9d4984935541eeefac4699"
format_stamp: "Formatted at 2026-08-12 05:29:47 on dist-test-slave-zpfg"
server_key: "6a2cdb3437b4791875df14ad5f94bf4c"
server_key_iv: "1e9832522ee698b2cbdaa6903b8b31cd"
server_key_version: "encryptionkey@0"
I20260812 05:29:47.723357 15933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:47.724162 15948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:47.724371 15933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:47.724488 15933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "97056a4d6a9d4984935541eeefac4699"
format_stamp: "Formatted at 2026-08-12 05:29:47 on dist-test-slave-zpfg"
server_key: "6a2cdb3437b4791875df14ad5f94bf4c"
server_key_iv: "1e9832522ee698b2cbdaa6903b8b31cd"
server_key_version: "encryptionkey@0"
I20260812 05:29:47.724567 15933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:47.741598 15933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:47.744179 15933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:47.744371 15933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:47.749028 15933 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:34087
I20260812 05:29:47.749068 16005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:34087 every 8 connection(s)
I20260812 05:29:47.749449 15933 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:47.750586 16006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:47.751884 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 15933
I20260812 05:29:47.752017 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:47.752211 24841 external_mini_cluster.cc:1428] Setting key 4006f11e1d9e53325ff53e8775be9566
I20260812 05:29:47.752557 16006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:29:47 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.66.125@KRBTEST.COM; defaulting to no policy
I20260812 05:29:47.763819 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.753698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:34087 (local address 127.0.0.1:45804)
0812 05:29:47.753910 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.753957 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:47.754558 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:47.754867 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:47.754869 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:47.755165 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:47.755329 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:47.755333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.756369 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.756370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:47.757165 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:47.757168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.757700 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.757701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:47.757769 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:47.758404 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:47.758420 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:47.760241 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:47.762209 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:47.762217 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:47.762219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:47.762519 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:47.762763 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:47.762764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:47.762764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:47.762860 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:47.763186 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:47.763200 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:47.763285 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 05:29:47.763370 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 05:29:47.763599 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":81,"thread_start_us":126,"threads_started":1}
I20260812 05:29:47.763849 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.753944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45804 (local address 127.24.66.126:34087)
0812 05:29:47.754108 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:47.754113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:47.754163 (+ 50us) server_negotiation.cc:407] Connection header received
0812 05:29:47.754732 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:47.754734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:47.754751 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:47.754846 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:47.755428 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.756266 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.757257 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.757572 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.757722 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:47.760362 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:47.760370 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:47.760370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:47.760377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:47.762121 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:47.762590 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:47.762593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:47.762593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:47.762676 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:47.762909 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:47.762909 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:47.762910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:47.763123 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:47.763183 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:47.763404 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 05:29:47.763521 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.125@KRBTEST.COM" created.
I20260812 05:29:47.764865 16006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:47.765255 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.765110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:35655 (local address 127.0.0.1:43070)
0812 05:29:47.765164 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.765218 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 05:29:47.765455 15949 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.66.125:35655: connect: Connection refused (error 111)
I20260812 05:29:47.765959 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.765845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:35655 (local address 127.0.0.1:43078)
0812 05:29:47.765896 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.765928 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 05:29:47.766202 16006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.125:35655: Network error: Client connection negotiation failed: client connection to 127.24.66.125:35655: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab.
Entry for principal kudu/127.24.66.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab.
I20260812 05:29:47.785615 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.125:35655
--webserver_interface=127.24.66.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab
--principal=kudu/127.24.66.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:34087,127.24.66.125:35655,127.24.66.124:34091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:29:47.815500 16006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } attempt: 1
I20260812 05:29:47.816119 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.815957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:35655 (local address 127.0.0.1:43092)
0812 05:29:47.816025 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.816072 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 05:29:47.816638 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.816529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:35655 (local address 127.0.0.1:43100)
0812 05:29:47.816578 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.816609 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 05:29:47.816776 16006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.125:35655: Network error: Client connection negotiation failed: client connection to 127.24.66.125:35655: connect: Connection refused (error 111)
W20260812 05:29:47.873906 16013 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:29:47.874137 16013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:47.874155 16013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:47.874177 16013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:47.875433 16013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:47.875478 16013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:47.875491 16013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:47.875504 16013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:47.875516 16013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:47.877144 16013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:34087,127.24.66.125:35655,127.24.66.124:34091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab
--principal=kudu/127.24.66.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.125:35655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.66.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:47.877413 16013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:47.877688 16013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:47.882853 16019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:47.882861 16018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:47.882874 16021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:47.883230 16013 server_base.cc:1034] running on GCE node
I20260812 05:29:47.883512 16013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:47.883863 16013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:47.884946 16006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } attempt: 2
I20260812 05:29:47.885046 16013 hybrid_clock.cc:648] HybridClock initialized: now 1786512587885031 us; error 47 us; skew 500 ppm
I20260812 05:29:47.885511 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.885316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:35655 (local address 127.0.0.1:43102)
0812 05:29:47.885435 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.885476 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260812 05:29:47.885999 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.885872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:35655 (local address 127.0.0.1:43112)
0812 05:29:47.885939 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.885967 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 05:29:47.886117 16006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.125:35655: Network error: Client connection negotiation failed: client connection to 127.24.66.125:35655: connect: Connection refused (error 111)
Aug 12 05:29:47 dist-test-slave-zpfg krb5kdc[15919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:47.888033 16013 init.cc:377] Logged in from keytab as kudu/127.24.66.125@KRBTEST.COM (short username kudu)
I20260812 05:29:47.888543 16013 webserver.cc:460] Webserver started at http://127.24.66.125:44827/ using document root <none> and password file <none>
I20260812 05:29:47.888804 16013 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:47.888852 16013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:47.889015 16013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:47.890122 16013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/data/instance:
uuid: "59b85b6851e14b5ea1ff815fb7ebdb35"
format_stamp: "Formatted at 2026-08-12 05:29:47 on dist-test-slave-zpfg"
server_key: "1ba7a98baf737beab884af3477a43b8e"
server_key_iv: "5cf94c94e38f5be7edc0ced518ff9ba0"
server_key_version: "encryptionkey@0"
I20260812 05:29:47.890523 16013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/wal/instance:
uuid: "59b85b6851e14b5ea1ff815fb7ebdb35"
format_stamp: "Formatted at 2026-08-12 05:29:47 on dist-test-slave-zpfg"
server_key: "1ba7a98baf737beab884af3477a43b8e"
server_key_iv: "5cf94c94e38f5be7edc0ced518ff9ba0"
server_key_version: "encryptionkey@0"
I20260812 05:29:47.891952 16013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:47.892767 16029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:47.892944 16013 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:47.893046 16013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/wal
uuid: "59b85b6851e14b5ea1ff815fb7ebdb35"
format_stamp: "Formatted at 2026-08-12 05:29:47 on dist-test-slave-zpfg"
server_key: "1ba7a98baf737beab884af3477a43b8e"
server_key_iv: "5cf94c94e38f5be7edc0ced518ff9ba0"
server_key_version: "encryptionkey@0"
I20260812 05:29:47.893131 16013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:47.906113 16013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:47.908414 16013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:47.908622 16013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:47.913512 16013 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.125:35655
I20260812 05:29:47.913568 16084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.125:35655 every 8 connection(s)
I20260812 05:29:47.914003 16013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
I20260812 05:29:47.915269 16085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:47.917079 16085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:47.922533 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 16013
I20260812 05:29:47.922670 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-1/wal/instance
I20260812 05:29:47.922894 24841 external_mini_cluster.cc:1428] Setting key 318d83a1855951c092ae851e5d8e11a4
Aug 12 05:29:47 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:47.927930 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.918091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45814 (local address 127.24.66.126:34087)
0812 05:29:47.918158 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:47.918161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:47.918416 (+ 255us) server_negotiation.cc:407] Connection header received
0812 05:29:47.919021 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:47.919025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:47.919034 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:47.919099 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:47.919814 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.920318 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.921439 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.921648 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.921702 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:47.924302 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:47.924305 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:47.924306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:47.924316 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:47.926060 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:47.926636 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:47.926637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:47.926638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:47.926700 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:47.927010 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:47.927011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:47.927011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:47.927137 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:47.927281 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:47.927777 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 05:29:47.927862 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:29:47.928122 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.918001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:34087 (local address 127.0.0.1:45814)
0812 05:29:47.918216 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.918265 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:47.918918 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:47.919190 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:47.919194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:47.919505 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:47.919725 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:47.919729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.920425 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.920426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:47.921351 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:47.921353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.921741 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.921742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:47.921871 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:47.922435 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:47.922449 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:47.924192 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:47.926157 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:47.926166 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:47.926168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:47.926548 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:47.926767 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:47.926768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:47.926769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:47.926932 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:47.927285 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:47.927294 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:47.927514 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 05:29:47.927636 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 05:29:47.927849 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":124,"threads_started":1}
I20260812 05:29:47.928857 16085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.66.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.124@KRBTEST.COM" created.
Aug 12 05:29:47 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:29:47.939870 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.929201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:35655 (local address 127.0.0.1:43120)
0812 05:29:47.929255 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.929267 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:47.929305 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:47.929587 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:47.929588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:47.929763 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:47.929846 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:47.929848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.931031 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.931033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:47.931913 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:47.931916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.932476 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.932477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:47.932566 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:47.933047 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:47.933052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:47.936378 (+ 3326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:47.938456 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:47.938458 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:47.938459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:47.938704 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:47.938969 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:47.938970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:47.938970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:47.939033 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:47.939391 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:47.939392 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:47.939432 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 05:29:47.939522 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 05:29:47.939657 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 05:29:47.939899 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.929256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43120 (local address 127.24.66.125:35655)
0812 05:29:47.929430 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:47.929434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:47.929458 (+ 24us) server_negotiation.cc:407] Connection header received
0812 05:29:47.929477 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:47.929479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:47.929490 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:47.929557 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:47.929922 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.930922 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.932017 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.932399 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.932460 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:47.936472 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:47.936485 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:47.936486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:47.936495 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:47.938324 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:47.938819 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:47.938822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:47.938822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:47.938890 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:47.939134 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:47.939135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:47.939136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:47.939313 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:47.939418 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:47.939558 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 05:29:47.939716 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":144,"thread_start_us":115,"threads_started":1}
I20260812 05:29:47.940949 16085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:47.941406 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.941234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52122)
0812 05:29:47.941332 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.941383 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260812 05:29:47.941574 16032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
I20260812 05:29:47.942168 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.942045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52128)
0812 05:29:47.942113 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.942142 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 05:29:47.942317 16085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:34091: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab.
Entry for principal kudu/127.24.66.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab.
I20260812 05:29:47.953400 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.124:34091
--webserver_interface=127.24.66.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab
--principal=kudu/127.24.66.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:34087,127.24.66.125:35655,127.24.66.124:34091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:29:47.977293 16006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } attempt: 3
Aug 12 05:29:47 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:29:47.985569 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.977867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:35655 (local address 127.0.0.1:43130)
0812 05:29:47.977933 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.977953 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:47.978026 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:47.978190 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:47.978191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:47.978392 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:47.978463 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:47.978465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.979406 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.979409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:47.980012 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:47.980015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.980446 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.980448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:47.980534 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:47.981087 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:47.981092 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:47.982328 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:47.984387 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:47.984388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:47.984388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:47.984582 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:47.984813 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:47.984814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:47.984814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:47.984876 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:47.985137 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:47.985139 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:47.985190 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 05:29:47.985309 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 05:29:47.985468 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 05:29:47.985569 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.977950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43130 (local address 127.24.66.125:35655)
0812 05:29:47.978028 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:47.978030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:47.978042 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:29:47.978094 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:47.978095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:47.978107 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:47.978182 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:47.978595 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.979135 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.980126 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:47.980365 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:47.980448 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:47.982482 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:47.982486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:47.982486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:47.982499 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:47.984146 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:47.984660 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:47.984662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:47.984662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:47.984726 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:47.984949 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:47.984950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:47.984950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:47.985069 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:47.985146 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:47.985339 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 05:29:47.985464 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 05:29:47.985967 16006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:47.986369 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.986238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52134)
0812 05:29:47.986291 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.986320 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 05:29:47.986774 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.986667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52140)
0812 05:29:47.986723 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.986759 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 05:29:47.986912 16006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:34091: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
I20260812 05:29:47.991600 16085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } attempt: 1
I20260812 05:29:47.992079 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.991956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52156)
0812 05:29:47.992011 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.992044 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 05:29:47.992499 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:47.992408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52168)
0812 05:29:47.992453 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:47.992478 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 05:29:47.992631 16085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:34091: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
I20260812 05:29:48.018075 16006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } attempt: 1
I20260812 05:29:48.018635 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.018500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52176)
0812 05:29:48.018563 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:48.018604 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 05:29:48.019038 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.018937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52178)
0812 05:29:48.018986 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:48.019013 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 05:29:48.019129 16006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:34091: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
W20260812 05:29:48.044329 16094 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:29:48.044596 16094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:48.044628 16094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:48.044672 16094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:48.045868 16094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:48.045944 16094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:48.045969 16094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:48.045990 16094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:48.046020 16094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:48.047663 16094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:34087,127.24.66.125:35655,127.24.66.124:34091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab
--principal=kudu/127.24.66.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.124:34091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.66.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:48.047962 16094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:48.048213 16094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:48.053315 16099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:48.053368 16100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:48.053328 16102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:48.053668 16094 server_base.cc:1034] running on GCE node
I20260812 05:29:48.054039 16094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:48.054410 16094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:48.055625 16094 hybrid_clock.cc:648] HybridClock initialized: now 1786512588055598 us; error 41 us; skew 500 ppm
Aug 12 05:29:48 dist-test-slave-zpfg krb5kdc[15919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512588, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:48.058435 16094 init.cc:377] Logged in from keytab as kudu/127.24.66.124@KRBTEST.COM (short username kudu)
I20260812 05:29:48.058918 16094 webserver.cc:460] Webserver started at http://127.24.66.124:45255/ using document root <none> and password file <none>
I20260812 05:29:48.059208 16094 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:48.059281 16094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:48.059434 16094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:48.060537 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/data/instance:
uuid: "a68d2e3320ce41c79b61934dbe4c3e2d"
format_stamp: "Formatted at 2026-08-12 05:29:48 on dist-test-slave-zpfg"
server_key: "169d0ff7922088165394708d3daff6bb"
server_key_iv: "df4948520ea422f09076a6352c6cae43"
server_key_version: "encryptionkey@0"
I20260812 05:29:48.060810 16085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } attempt: 2
I20260812 05:29:48.060945 16094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/wal/instance:
uuid: "a68d2e3320ce41c79b61934dbe4c3e2d"
format_stamp: "Formatted at 2026-08-12 05:29:48 on dist-test-slave-zpfg"
server_key: "169d0ff7922088165394708d3daff6bb"
server_key_iv: "df4948520ea422f09076a6352c6cae43"
server_key_version: "encryptionkey@0"
I20260812 05:29:48.061286 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.061118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52194)
0812 05:29:48.061194 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:48.061238 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 05:29:48.061655 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.061558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52210)
0812 05:29:48.061608 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:48.061635 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 05:29:48.061792 16085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:34091: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
I20260812 05:29:48.062739 16094 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 05:29:48.063620 16113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:48.063844 16094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:48.063942 16094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/wal
uuid: "a68d2e3320ce41c79b61934dbe4c3e2d"
format_stamp: "Formatted at 2026-08-12 05:29:48 on dist-test-slave-zpfg"
server_key: "169d0ff7922088165394708d3daff6bb"
server_key_iv: "df4948520ea422f09076a6352c6cae43"
server_key_version: "encryptionkey@0"
I20260812 05:29:48.064015 16094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:48.093360 16094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:48.094275 16006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } attempt: 2
I20260812 05:29:48.094779 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.094618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52214)
0812 05:29:48.094689 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:48.094733 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 05:29:48.095423 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.095106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52220)
0812 05:29:48.095345 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:48.095385 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204}
W20260812 05:29:48.095571 16006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:34091: Network error: Client connection negotiation failed: client connection to 127.24.66.124:34091: connect: Connection refused (error 111)
I20260812 05:29:48.096151 16094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:48.096315 16094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:48.100696 16094 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.124:34091
I20260812 05:29:48.100732 16169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.124:34091 every 8 connection(s)
I20260812 05:29:48.101135 16094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
I20260812 05:29:48.102208 16170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:48.104039 16170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:48.109902 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 16094
I20260812 05:29:48.110018 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786512301961891-24841-0/minicluster-data/master-2/wal/instance
I20260812 05:29:48.110204 24841 external_mini_cluster.cc:1428] Setting key 3cb725ddb80aa23c79be5aa71785dc91
Aug 12 05:29:48 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512588, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:48.115188 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.104970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45822 (local address 127.24.66.126:34087)
0812 05:29:48.105087 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.105090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.105314 (+ 224us) server_negotiation.cc:407] Connection header received
0812 05:29:48.105775 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.105776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.105787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.105856 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.106609 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.107085 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.108075 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.108320 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.108381 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.111527 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:48.111529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:48.111529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:48.111535 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:48.113604 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.114095 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.114096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.114096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.114167 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.114464 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.114465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.114466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.114607 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:48.114659 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:48.115055 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 05:29:48.115122 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 05:29:48.115424 16172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.104951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:34087 (local address 127.0.0.1:45822)
0812 05:29:48.105170 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:48.105214 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:48.105685 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:48.105936 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:48.105940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:48.106201 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:48.106507 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:48.106514 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.107168 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.107169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:48.107985 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:48.107987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.108413 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.108414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:48.108521 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.109401 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:48.109414 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:48.111442 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:48.113689 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:48.113696 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:48.113698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:48.114021 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:48.114241 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:48.114242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:48.114242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:48.114393 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:48.114678 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:48.114686 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:48.114878 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 05:29:48.114993 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 05:29:48.115158 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":135,"threads_started":1}
I20260812 05:29:48.116055 16170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:29:48 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
I20260812 05:29:48.120286 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.110646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52222 (local address 127.24.66.124:34091)
0812 05:29:48.110814 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.110819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.110879 (+ 60us) server_negotiation.cc:407] Connection header received
0812 05:29:48.110967 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.110970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.110992 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.111082 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.111885 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.112985 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.113900 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.114316 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.114439 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.117218 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:48.117225 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:48.117225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:48.117231 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:48.119057 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.119468 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.119471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.119472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.119526 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.119715 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.119716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.119716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.119909 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:48.120053 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:48.120147 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 05:29:48.120208 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
Aug 12 05:29:48 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512588, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:29:48.124207 16172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.116340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:35655 (local address 127.0.0.1:43140)
0812 05:29:48.116397 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:48.116409 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:48.116457 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:48.116575 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:48.116576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:48.116723 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:48.116790 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:48.116792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.117573 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.117574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:48.118469 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:48.118472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.118872 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.118873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:48.118955 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.119374 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:48.119378 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:48.120993 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:48.123169 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:48.123170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:48.123170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:48.123344 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:48.123551 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:48.123552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:48.123553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:48.123622 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:48.123884 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:48.123885 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:48.123926 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 05:29:48.124012 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 05:29:48.124113 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 05:29:48.124207 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.116395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.24.66.125:35655)
0812 05:29:48.116439 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.116441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.116450 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:29:48.116517 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.116518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.116527 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.116571 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.116856 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.117502 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.118555 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.118792 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.118857 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.121175 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:48.121176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:48.121177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:48.121184 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:48.123069 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.123419 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.123421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.123421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.123479 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.123693 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.123694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.123694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.123810 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:48.123879 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:48.124038 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 05:29:48.124113 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:29:48.125030 16170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 05:29:48 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
Aug 12 05:29:48 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512588, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
I20260812 05:29:48.129076 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.121118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45832 (local address 127.24.66.126:34087)
0812 05:29:48.121158 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.121160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.121169 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:29:48.121216 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.121217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.121228 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.121292 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.121722 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.122447 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.123191 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.123438 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.123512 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.126042 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:48.126044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:48.126048 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:48.126055 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:48.127763 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.128131 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.128132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.128133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.128190 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.128394 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.128395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.128396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.128511 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:48.128695 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:48.128733 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 05:29:48.128822 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:29:48.131772 16172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.125299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52232)
0812 05:29:48.125350 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:48.125361 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:48.125405 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:48.125523 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:48.125524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:48.125669 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:48.125704 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:48.125706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.126330 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.126331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:48.127001 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:48.127003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.127474 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.127475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:48.127515 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.127933 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:48.127937 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:48.129078 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:48.130755 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:48.130756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:48.130756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:48.130920 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:48.131137 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:48.131138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:48.131138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:48.131197 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:48.131449 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:48.131449 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:48.131486 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 05:29:48.131556 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 05:29:48.131657 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 05:29:48.131819 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.125351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.24.66.124:34091)
0812 05:29:48.125388 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.125390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.125398 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:29:48.125445 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.125446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.125456 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.125521 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.125809 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.126262 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.127078 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.127392 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.127469 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.129173 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:48.129175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:48.129175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:48.129182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:48.130682 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.131015 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.131016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.131016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.131072 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.131258 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.131259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.131260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.131381 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:48.131456 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:48.131580 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 05:29:48.131704 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":19}
I20260812 05:29:48.133313 16170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d: Bootstrap starting.
I20260812 05:29:48.134260 16170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:48.134605 16170 log.cc:824] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d: Log is configured to *not* fsync() on all Append() calls
Aug 12 05:29:48 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:29:48.135489 16170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d: No bootstrap required, opened a new log
I20260812 05:29:48.137290 16170 raft_consensus.cc:348] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } }
I20260812 05:29:48.137423 16170 raft_consensus.cc:729] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a68d2e3320ce41c79b61934dbe4c3e2d, State: Initialized, Role: FOLLOWER
I20260812 05:29:48.137521 16170 consensus_queue.cc:260] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } }
I20260812 05:29:48.137871 16176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } } }
I20260812 05:29:48.137950 16170 sys_catalog.cc:564] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:48.138013 16176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:29:48.138655 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.130484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.24.66.125:35655)
0812 05:29:48.130554 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.130556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.130826 (+ 270us) server_negotiation.cc:407] Connection header received
0812 05:29:48.130899 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.130900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.130911 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.130967 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.131449 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.132228 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.132920 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.133147 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.133224 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.135695 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:48.135696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:48.135697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:48.135705 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:48.137270 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.137686 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.137687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.137687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.137748 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.137987 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.137988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.137989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.138160 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:48.138287 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:48.138448 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 05:29:48.138551 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:29:48.140259 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 05:29:48.140208 16188 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:48.141064 16188 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:48.141165 15977 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "97056a4d6a9d4984935541eeefac4699"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45832
I20260812 05:29:48.140477 16094 master_runner.cc:186] Error getting master registration for 127.24.66.126:34087: OK, instance_id { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" instance_seqno: 1786512587747704 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:29:48.143492 16094 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 05:29:48.152956 16085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } attempt: 3
Aug 12 05:29:48 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
I20260812 05:29:48.161855 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.153408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52234)
0812 05:29:48.153476 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:48.153492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:48.153566 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:48.153722 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:48.153723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:48.153953 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:48.154013 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:48.154015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.155007 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.155008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:48.155832 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:48.155834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.156351 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.156352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:48.156413 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.156955 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:48.156961 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:48.158676 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:48.160599 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:48.160601 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:48.160601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:48.160841 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:48.161055 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:48.161056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:48.161057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:48.161128 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:48.161502 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:48.161503 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:48.161554 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 05:29:48.161628 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 05:29:48.161742 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 05:29:48.161866 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.153499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.24.66.124:34091)
0812 05:29:48.153574 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.153576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.153589 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:29:48.153618 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.153619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.153630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.153691 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.154095 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.154896 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.155926 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.156256 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.156341 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.158789 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:48.158791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:48.158792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:48.158800 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:48.160499 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.160923 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.160924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.160924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.160985 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.161194 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.161195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.161195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.161404 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:48.161508 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:48.161666 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 05:29:48.161757 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:29:48.163635 16085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35: Bootstrap starting.
I20260812 05:29:48.164336 16085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:48.164669 16085 log.cc:824] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:48.165494 16085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35: No bootstrap required, opened a new log
I20260812 05:29:48.167294 16085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } }
I20260812 05:29:48.167446 16085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59b85b6851e14b5ea1ff815fb7ebdb35, State: Initialized, Role: FOLLOWER
I20260812 05:29:48.167541 16085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } }
I20260812 05:29:48.167959 16191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } } }
I20260812 05:29:48.168052 16085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:48.168059 16191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 05:29:48.170009 16202 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:48.170069 16202 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:48.170404 16013 master_runner.cc:186] Error getting master registration for 127.24.66.126:34087: OK, instance_id { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" instance_seqno: 1786512587747704 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:29:48.173462 16013 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 05:29:48.194736 16006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } attempt: 3
Aug 12 05:29:48 dist-test-slave-zpfg krb5kdc[15919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512587, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
I20260812 05:29:48.204157 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.195161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:34091 (local address 127.0.0.1:52246)
0812 05:29:48.195231 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:48.195243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:48.195289 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:48.195492 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:48.195493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:48.195704 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:48.195767 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:48.195769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.196863 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.196865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:48.197754 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:48.197757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.198298 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.198299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:48.198384 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.198989 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:48.198995 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:48.200783 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:48.202954 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:48.202955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:48.202955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:48.203196 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:48.203415 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:48.203415 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:48.203416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:48.203476 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:48.203800 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:48.203801 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:48.203850 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 05:29:48.203922 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 05:29:48.204034 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 05:29:48.204782 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.195245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52246 (local address 127.24.66.124:34091)
0812 05:29:48.195297 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.195300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.195316 (+ 16us) server_negotiation.cc:407] Connection header received
0812 05:29:48.195364 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.195365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.195377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.195485 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.195851 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.196745 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.197870 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.198168 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.198232 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.200892 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:48.200894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:48.200894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:48.200903 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:48.202851 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.203282 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.203283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.203283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.203341 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.203545 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.203546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.203546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.203705 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:48.203771 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:48.204569 (+ 798us) server_negotiation.cc:299] Negotiation successful
0812 05:29:48.204679 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 05:29:48.207006 16006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699: Bootstrap starting.
I20260812 05:29:48.207868 16006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:48.208235 16006 log.cc:824] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:48.209260 16006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699: No bootstrap required, opened a new log
I20260812 05:29:48.210988 16006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } }
I20260812 05:29:48.211239 16006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97056a4d6a9d4984935541eeefac4699, State: Initialized, Role: FOLLOWER
I20260812 05:29:48.211467 16006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } }
I20260812 05:29:48.211992 16205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } } }
I20260812 05:29:48.212096 16205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:29:48.212307 15977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 05:29:48.212394 15977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:48.213261 15977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } }
I20260812 05:29:48.213526 15977 leader_election.cc:288] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [CANDIDATE]: Term 1 election: Requested vote from peers 59b85b6851e14b5ea1ff815fb7ebdb35 (127.24.66.125:35655), a68d2e3320ce41c79b61934dbe4c3e2d (127.24.66.124:34091)
I20260812 05:29:48.213910 16144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97056a4d6a9d4984935541eeefac4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d"
I20260812 05:29:48.214036 16144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:48.213948 16057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97056a4d6a9d4984935541eeefac4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35"
I20260812 05:29:48.214131 16057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:48.214933 16144 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97056a4d6a9d4984935541eeefac4699 in term 1.
I20260812 05:29:48.215195 15951 leader_election.cc:302] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97056a4d6a9d4984935541eeefac4699, a68d2e3320ce41c79b61934dbe4c3e2d; no voters:
I20260812 05:29:48.212263 16006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:48.216010 16057 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97056a4d6a9d4984935541eeefac4699 in term 1.
I20260812 05:29:48.217067 16205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:48.217576 16205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [term 1 LEADER]: Becoming Leader. State: Replica: 97056a4d6a9d4984935541eeefac4699, State: Running, Role: LEADER
I20260812 05:29:48.217694 16205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } }
I20260812 05:29:48.218308 16205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97056a4d6a9d4984935541eeefac4699. Latest consensus state: current_term: 1 leader_uuid: "97056a4d6a9d4984935541eeefac4699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } } }
I20260812 05:29:48.218420 16205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [sys.catalog]: This master's current role is: LEADER
W20260812 05:29:48.219550 16223 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:48.219609 16223 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:48.221210 16229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:48.221428 16229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:48.223145 16144 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [term 1 FOLLOWER]: Refusing update from remote peer 97056a4d6a9d4984935541eeefac4699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:48.223150 16057 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [term 1 FOLLOWER]: Refusing update from remote peer 97056a4d6a9d4984935541eeefac4699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:48.223487 16205 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:48.223680 16205 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:48.225026 16191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97056a4d6a9d4984935541eeefac4699. Latest consensus state: current_term: 1 leader_uuid: "97056a4d6a9d4984935541eeefac4699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } } }
I20260812 05:29:48.225122 16191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:29:48.226289 16176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97056a4d6a9d4984935541eeefac4699. Latest consensus state: current_term: 1 leader_uuid: "97056a4d6a9d4984935541eeefac4699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } } }
I20260812 05:29:48.226442 16176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:29:48.227423 16205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97056a4d6a9d4984935541eeefac4699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } } }
I20260812 05:29:48.227533 16205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:48.227686 16230 mvcc.cc:205] Tried to move back new op lower bound from 7317555561359228928 to 7317555561340747776. Current Snapshot: MvccSnapshot[applied={T|T < 7317555561359228928}]
I20260812 05:29:48.228183 16176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97056a4d6a9d4984935541eeefac4699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } } }
I20260812 05:29:48.228271 16176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:29:48.228474 16233 mvcc.cc:205] Tried to move back new op lower bound from 7317555561359228928 to 7317555561340747776. Current Snapshot: MvccSnapshot[applied={T|T < 7317555561359228928}]
I20260812 05:29:48.229007 16229 catalog_manager.cc:1347] Generated new cluster ID: 5eea91d3e2ea4584bec6c46cd874ed43
I20260812 05:29:48.229053 16229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:48.232520 16009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.216531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45842 (local address 127.24.66.126:34087)
0812 05:29:48.216572 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.216575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.217495 (+ 920us) server_negotiation.cc:407] Connection header received
0812 05:29:48.217599 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.217600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.217610 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.217642 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.219420 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.220190 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.223083 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.223384 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.223427 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.227013 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:48.227015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:48.227015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:48.227024 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:48.228756 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.231363 (+ 2607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.231364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.231365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.231424 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.231617 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.231618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.231618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.231764 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:48.231846 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:48.232332 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 05:29:48.232423 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 05:29:48.234076 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.216906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52260 (local address 127.24.66.124:34091)
0812 05:29:48.216945 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.216947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.218553 (+ 1606us) server_negotiation.cc:407] Connection header received
0812 05:29:48.218648 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.218649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.218659 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.218688 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.221322 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.222038 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.224120 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.224413 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.224458 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.229169 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:48.229171 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:48.229172 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:48.229180 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:48.230957 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.232037 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.232038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.232039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.232104 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.233374 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.233375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.233375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.233520 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:48.233601 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:48.233900 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 05:29:48.233984 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 05:29:48.242811 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:48.217231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43164 (local address 127.24.66.125:35655)
0812 05:29:48.217271 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.217272 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.232772 (+ 15500us) server_negotiation.cc:407] Connection header received
0812 05:29:48.234233 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.234235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.234249 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.234280 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.235700 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.236477 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.237581 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:48.237849 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:48.237918 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:48.239654 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:48.239656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:48.239656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:48.239665 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:48.241425 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.241874 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.241875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.241876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.241934 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:48.242134 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:48.242135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:48.242135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:48.242264 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:48.242402 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:48.242620 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 05:29:48.242697 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:29:48.246210 16229 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:48.247046 16229 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:48.247177 16205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97056a4d6a9d4984935541eeefac4699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } } }
I20260812 05:29:48.247282 16205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:48.247857 16191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97056a4d6a9d4984935541eeefac4699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97056a4d6a9d4984935541eeefac4699" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34087 } } peers { permanent_uuid: "59b85b6851e14b5ea1ff815fb7ebdb35" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 35655 } } peers { permanent_uuid: "a68d2e3320ce41c79b61934dbe4c3e2d" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 34091 } } }
I20260812 05:29:48.247960 16191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:29:48.257287 16229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97056a4d6a9d4984935541eeefac4699: Generated new TSK 0
I20260812 05:29:48.257584 16229 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 05:29:48.282608 16009 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.281094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45858 (local address 127.24.66.126:34087)
0812 05:29:48.281142 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.281144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.281856 (+ 712us) server_negotiation.cc:407] Connection header received
0812 05:29:48.281971 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.281973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.281985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.282016 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.282530 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45858: BlockingRecv error: recv got EOF from 127.0.0.1:45858 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 05:29:48.283363 16174 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.281519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52268 (local address 127.24.66.124:34091)
0812 05:29:48.281561 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.281563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.282866 (+ 1303us) server_negotiation.cc:407] Connection header received
0812 05:29:48.282963 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.282964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.282974 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.283003 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.283302 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52268: BlockingRecv error: recv got EOF from 127.0.0.1:52268 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 05:29:48.284384 16090 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.283750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43178 (local address 127.24.66.125:35655)
0812 05:29:48.283790 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.283792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.283903 (+ 111us) server_negotiation.cc:407] Connection header received
0812 05:29:48.283995 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.283996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.284005 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.284033 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.284324 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43178: BlockingRecv error: recv got EOF from 127.0.0.1:43178 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 05:29:48.286808 16090 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.286017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.24.66.125:35655)
0812 05:29:48.286057 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.286059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.286241 (+ 182us) server_negotiation.cc:407] Connection header received
0812 05:29:48.286373 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.286374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.286392 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.286421 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.286762 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43192: BlockingRecv error: recv got EOF from 127.0.0.1:43192 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 05:29:48.289929 16090 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.289131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43200 (local address 127.24.66.125:35655)
0812 05:29:48.289167 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.289168 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.289396 (+ 228us) server_negotiation.cc:407] Connection header received
0812 05:29:48.289490 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.289491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.289504 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.289532 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.289883 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43200: BlockingRecv error: recv got EOF from 127.0.0.1:43200 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 05:29:48.294193 16090 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.293318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.24.66.125:35655)
0812 05:29:48.293361 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.293363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.293610 (+ 247us) server_negotiation.cc:407] Connection header received
0812 05:29:48.293725 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.293726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.293738 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.293769 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.294136 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43212: BlockingRecv error: recv got EOF from 127.0.0.1:43212 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 05:29:48.300696 16090 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.299764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43228 (local address 127.24.66.125:35655)
0812 05:29:48.299810 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.299812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.300081 (+ 269us) server_negotiation.cc:407] Connection header received
0812 05:29:48.300192 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.300193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.300205 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.300237 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.300636 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43228: BlockingRecv error: recv got EOF from 127.0.0.1:43228 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 05:29:48.311244 16090 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.310199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43232 (local address 127.24.66.125:35655)
0812 05:29:48.310248 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.310251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.310602 (+ 351us) server_negotiation.cc:407] Connection header received
0812 05:29:48.310737 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.310739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.310752 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.310785 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.311180 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43232: BlockingRecv error: recv got EOF from 127.0.0.1:43232 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 05:29:48.329959 16090 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.328928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43242 (local address 127.24.66.125:35655)
0812 05:29:48.328975 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.328977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.329256 (+ 279us) server_negotiation.cc:407] Connection header received
0812 05:29:48.329370 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.329371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.329384 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.329415 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.329897 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43242: BlockingRecv error: recv got EOF from 127.0.0.1:43242 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 05:29:48.364214 16090 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.363486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43254 (local address 127.24.66.125:35655)
0812 05:29:48.363546 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.363549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.363690 (+ 141us) server_negotiation.cc:407] Connection header received
0812 05:29:48.363723 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.363724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.363738 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.363801 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.364148 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43254: BlockingRecv error: recv got EOF from 127.0.0.1:43254 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 05:29:48.430109 16090 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.429443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43270 (local address 127.24.66.125:35655)
0812 05:29:48.429544 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.429545 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.429592 (+ 47us) server_negotiation.cc:407] Connection header received
0812 05:29:48.429618 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.429619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.429630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.429674 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.430033 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43270: BlockingRecv error: recv got EOF from 127.0.0.1:43270 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 05:29:48.560393 16090 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.559566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43276 (local address 127.24.66.125:35655)
0812 05:29:48.559681 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.559683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.559696 (+ 13us) server_negotiation.cc:407] Connection header received
0812 05:29:48.559778 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.559779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.559790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.559853 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.560298 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43276: BlockingRecv error: recv got EOF from 127.0.0.1:43276 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260812 05:29:48.818199 16090 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:48.817587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.24.66.125:35655)
0812 05:29:48.817663 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:48.817665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:48.817691 (+ 26us) server_negotiation.cc:407] Connection header received
0812 05:29:48.817740 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:48.817741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:48.817752 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:48.817798 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:48.818148 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43282: BlockingRecv error: recv got EOF from 127.0.0.1:43282 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 05:29:49.141665 16188 catalog_manager.cc:1259] Loaded cluster ID: 5eea91d3e2ea4584bec6c46cd874ed43
I20260812 05:29:49.141812 16188 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d: loading cluster ID for follower catalog manager: success
I20260812 05:29:49.142760 16188 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d: acquiring CA information for follower catalog manager: success
I20260812 05:29:49.143103 16188 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a68d2e3320ce41c79b61934dbe4c3e2d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:29:49.170593 16202 catalog_manager.cc:1259] Loaded cluster ID: 5eea91d3e2ea4584bec6c46cd874ed43
I20260812 05:29:49.170682 16202 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35: loading cluster ID for follower catalog manager: success
I20260812 05:29:49.171566 16202 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35: acquiring CA information for follower catalog manager: success
I20260812 05:29:49.171916 16202 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 59b85b6851e14b5ea1ff815fb7ebdb35: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:29:49.334470 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:49.331733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43292 (local address 127.24.66.125:35655)
0812 05:29:49.331907 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:49.331908 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:49.331917 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:29:49.331944 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:49.331945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:49.331997 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:49.332073 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:49.332251 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:49.332793 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:49.333637 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:49.333847 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:49.333904 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:49.334094 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:49.334208 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:49.334268 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:49.334327 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 05:29:49.334400 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":122,"threads_started":1}
I20260812 05:29:49.339825 16313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:49.336689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52276 (local address 127.24.66.124:34091)
0812 05:29:49.336824 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:49.336826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:49.336838 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:29:49.336890 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:49.336891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:49.336949 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:49.337042 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:49.337202 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:49.337833 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:49.338875 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:49.339129 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:49.339221 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:49.339412 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:49.339534 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:49.339651 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:49.339695 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 05:29:49.339719 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":53,"threads_started":1}
I20260812 05:29:49.344480 16313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:49.341681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52284 (local address 127.24.66.124:34091)
0812 05:29:49.341727 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:49.341730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:49.341965 (+ 235us) server_negotiation.cc:407] Connection header received
0812 05:29:49.342063 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:49.342064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:49.342075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:49.342133 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:49.342237 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:49.342791 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:49.343662 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:49.343983 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:49.344057 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:49.344206 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:49.344244 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:49.344312 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:49.344371 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 05:29:49.344393 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 05:29:49.345242 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:49.341620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45860 (local address 127.24.66.126:34087)
0812 05:29:49.341726 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:49.341728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:49.341995 (+ 267us) server_negotiation.cc:407] Connection header received
0812 05:29:49.342044 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:49.342045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:49.342101 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:49.342177 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:49.342382 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:49.343219 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:49.344117 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:49.344466 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:49.344529 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:49.344896 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:49.344983 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:49.345070 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:49.345126 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 05:29:49.345148 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":57,"threads_started":1}
I20260812 05:29:49.345590 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:49.341735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.24.66.125:35655)
0812 05:29:49.341777 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:49.341779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:49.341937 (+ 158us) server_negotiation.cc:407] Connection header received
0812 05:29:49.341988 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:49.341989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:49.342003 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:49.342058 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:49.342267 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:49.343136 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:49.344464 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:49.344776 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:49.344824 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:49.345294 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:49.345328 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:49.345405 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:49.345477 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 05:29:49.345502 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:29:49.347456 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 15933
I20260812 05:29:49.356171 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 16013
I20260812 05:29:49.362636 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 16094
2026-08-12T05:29:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3073 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 05:29:49 dist-test-slave-zpfg krb5kdc[16325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 05:29:49 dist-test-slave-zpfg krb5kdc[16325](info): set up 2 sockets
Aug 12 05:29:49 dist-test-slave-zpfg krb5kdc[16325](info): commencing operation
krb5kdc: starting...
W20260812 05:29:50.497040 24841 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/test-user.keytab.
Aug 12 05:29:50 dist-test-slave-zpfg krb5kdc[16325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T05:29:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T05:29:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.66.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
Entry for principal kudu/127.24.66.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab.
I20260812 05:29:50.633616 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:38491
--webserver_interface=127.24.66.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:38491,127.24.66.125:42767,127.24.66.124:42277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 05:29:50.718670 16339 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:29:50.718928 16339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:50.718958 16339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:50.719003 16339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:50.720151 16339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:50.720219 16339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:50.720244 16339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:50.720284 16339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:50.720335 16339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:50.721851 16339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:38491,127.24.66.125:42767,127.24.66.124:42277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.126.keytab
--principal=kudu/127.24.66.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:38491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:50.722136 16339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:50.722440 16339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:50.727537 16344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:50.727516 16347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:50.727612 16345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:50.727649 16339 server_base.cc:1034] running on GCE node
I20260812 05:29:50.728106 16339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:50.728425 16339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:50.729602 16339 hybrid_clock.cc:648] HybridClock initialized: now 1786512590729573 us; error 48 us; skew 500 ppm
Aug 12 05:29:50 dist-test-slave-zpfg krb5kdc[16325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:50.732535 16339 init.cc:377] Logged in from keytab as kudu/127.24.66.126@KRBTEST.COM (short username kudu)
I20260812 05:29:50.733085 16339 webserver.cc:460] Webserver started at http://127.24.66.126:42237/ using document root <none> and password file <none>
I20260812 05:29:50.733353 16339 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:50.733424 16339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:50.733583 16339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:50.734860 16339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/data/instance:
uuid: "c551664f2e22459896ae6ab0cd1fe994"
format_stamp: "Formatted at 2026-08-12 05:29:50 on dist-test-slave-zpfg"
server_key: "2066428419c513d8e7f135389f98792e"
server_key_iv: "2ec118d431066fa58d61c0456de69168"
server_key_version: "encryptionkey@0"
I20260812 05:29:50.735232 16339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/wal/instance:
uuid: "c551664f2e22459896ae6ab0cd1fe994"
format_stamp: "Formatted at 2026-08-12 05:29:50 on dist-test-slave-zpfg"
server_key: "2066428419c513d8e7f135389f98792e"
server_key_iv: "2ec118d431066fa58d61c0456de69168"
server_key_version: "encryptionkey@0"
I20260812 05:29:50.736821 16339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 05:29:50.737733 16355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:50.738080 16339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 05:29:50.738323 16339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/wal
uuid: "c551664f2e22459896ae6ab0cd1fe994"
format_stamp: "Formatted at 2026-08-12 05:29:50 on dist-test-slave-zpfg"
server_key: "2066428419c513d8e7f135389f98792e"
server_key_iv: "2ec118d431066fa58d61c0456de69168"
server_key_version: "encryptionkey@0"
I20260812 05:29:50.738477 16339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:50.758231 16339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:50.760905 16339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:50.761117 16339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:50.766280 16339 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:38491
I20260812 05:29:50.766408 16410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:38491 every 8 connection(s)
I20260812 05:29:50.766772 16339 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/data/info.pb
I20260812 05:29:50.767874 16412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:50.769448 16412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:50.769677 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 16339
I20260812 05:29:50.769784 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-0/wal/instance
I20260812 05:29:50.770052 24841 external_mini_cluster.cc:1428] Setting key 0a4c68ae33ef39f2cddb1f12b5b25304
Aug 12 05:29:50 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.66.125@KRBTEST.COM; defaulting to no policy
I20260812 05:29:50.784089 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.770269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38491 (local address 127.0.0.1:50798)
0812 05:29:50.770523 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.770586 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:50.771092 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:50.771493 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:50.771497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:50.771941 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:50.772223 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:50.772229 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.774017 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.774018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:50.775398 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:50.775405 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.776252 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.776253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:50.776350 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:50.777160 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:50.777171 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:50.779403 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:50.781953 (+ 2550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:50.781962 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:50.781964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:50.782373 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:50.782716 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:50.782717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:50.782718 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:50.782859 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:50.783416 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:50.783422 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:50.783498 (+ 76us) client_negotiation.cc:769] Sending connection context
0812 05:29:50.783593 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 05:29:50.783801 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":143,"threads_started":1}
I20260812 05:29:50.784096 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.770418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50798 (local address 127.24.66.126:38491)
0812 05:29:50.771253 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:50.771257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:50.771269 (+ 12us) server_negotiation.cc:407] Connection header received
0812 05:29:50.771333 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:50.771334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:50.771350 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:50.771414 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:50.772347 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.773906 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.775524 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.776082 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.776253 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:50.779493 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:50.779501 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:50.779502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:50.779509 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:50.781817 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:50.782485 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:50.782490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:50.782490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:50.782611 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:50.782943 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:50.782944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:50.782945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:50.783275 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:50.783439 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:50.783665 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 05:29:50.783853 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":809,"thread_start_us":92,"threads_started":1}
I20260812 05:29:50.785038 16412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.125@KRBTEST.COM" created.
I20260812 05:29:50.785441 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.785311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:42767 (local address 127.0.0.1:36822)
0812 05:29:50.785355 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.785409 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260812 05:29:50.785588 16356 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.66.125:42767: connect: Connection refused (error 111)
I20260812 05:29:50.786087 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.785981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:42767 (local address 127.0.0.1:36824)
0812 05:29:50.786036 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.786062 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 05:29:50.786204 16412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.125:42767: Network error: Client connection negotiation failed: client connection to 127.24.66.125:42767: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab.
Entry for principal kudu/127.24.66.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab.
I20260812 05:29:50.803743 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.125:42767
--webserver_interface=127.24.66.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab
--principal=kudu/127.24.66.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:38491,127.24.66.125:42767,127.24.66.124:42277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:29:50.835464 16412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } attempt: 1
I20260812 05:29:50.836068 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.835915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:42767 (local address 127.0.0.1:36830)
0812 05:29:50.835988 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.836037 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 05:29:50.836508 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.836406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:42767 (local address 127.0.0.1:36838)
0812 05:29:50.836453 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.836480 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 05:29:50.836622 16412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.125:42767: Network error: Client connection negotiation failed: client connection to 127.24.66.125:42767: connect: Connection refused (error 111)
W20260812 05:29:50.891211 16420 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:29:50.891451 16420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:50.891471 16420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:50.891494 16420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:50.892738 16420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:50.892786 16420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:50.892802 16420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:50.892814 16420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:50.892827 16420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:50.894449 16420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:38491,127.24.66.125:42767,127.24.66.124:42277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.125.keytab
--principal=kudu/127.24.66.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.125:42767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.66.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:50.894728 16420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:50.895001 16420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:50.900549 16427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:50.900620 16420 server_base.cc:1034] running on GCE node
W20260812 05:29:50.900535 16430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:50.900535 16426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:50.901000 16420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:50.901288 16420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:50.902467 16420 hybrid_clock.cc:648] HybridClock initialized: now 1786512590902450 us; error 47 us; skew 500 ppm
I20260812 05:29:50.904771 16412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } attempt: 2
Aug 12 05:29:50 dist-test-slave-zpfg krb5kdc[16325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:50.905383 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.905234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:42767 (local address 127.0.0.1:36852)
0812 05:29:50.905305 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.905345 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 05:29:50.905512 16420 init.cc:377] Logged in from keytab as kudu/127.24.66.125@KRBTEST.COM (short username kudu)
I20260812 05:29:50.905901 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.905785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:42767 (local address 127.0.0.1:36860)
0812 05:29:50.905837 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.905872 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.125:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 05:29:50.906009 16412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.125:42767: Network error: Client connection negotiation failed: client connection to 127.24.66.125:42767: connect: Connection refused (error 111)
I20260812 05:29:50.906258 16420 webserver.cc:460] Webserver started at http://127.24.66.125:46211/ using document root <none> and password file <none>
I20260812 05:29:50.906560 16420 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:50.906620 16420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:50.906777 16420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:50.907963 16420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/data/instance:
uuid: "c815b72bb62d4b7e9d78fec26f9d5526"
format_stamp: "Formatted at 2026-08-12 05:29:50 on dist-test-slave-zpfg"
server_key: "7ae926a9de8ac237a79328d30be75a87"
server_key_iv: "89ead8bf936812289b08ba728e47db3c"
server_key_version: "encryptionkey@0"
I20260812 05:29:50.908354 16420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/wal/instance:
uuid: "c815b72bb62d4b7e9d78fec26f9d5526"
format_stamp: "Formatted at 2026-08-12 05:29:50 on dist-test-slave-zpfg"
server_key: "7ae926a9de8ac237a79328d30be75a87"
server_key_iv: "89ead8bf936812289b08ba728e47db3c"
server_key_version: "encryptionkey@0"
I20260812 05:29:50.909833 16420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 05:29:50.910858 16437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:50.911257 16420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 05:29:50.911348 16420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/wal
uuid: "c815b72bb62d4b7e9d78fec26f9d5526"
format_stamp: "Formatted at 2026-08-12 05:29:50 on dist-test-slave-zpfg"
server_key: "7ae926a9de8ac237a79328d30be75a87"
server_key_iv: "89ead8bf936812289b08ba728e47db3c"
server_key_version: "encryptionkey@0"
I20260812 05:29:50.911448 16420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:50.922798 16420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:50.925454 16420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:50.925626 16420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:50.930179 16420 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.125:42767
I20260812 05:29:50.930217 16495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.125:42767 every 8 connection(s)
I20260812 05:29:50.930675 16420 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/data/info.pb
I20260812 05:29:50.931912 16496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:50.933645 16496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:50.940028 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 16420
I20260812 05:29:50.940131 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-1/wal/instance
I20260812 05:29:50.940346 24841 external_mini_cluster.cc:1428] Setting key 50c30c83f4a0e81d8db902f921cd70ad
Aug 12 05:29:50 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:50.944615 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.934487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50814 (local address 127.24.66.126:38491)
0812 05:29:50.934556 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:50.934558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:50.934877 (+ 319us) server_negotiation.cc:407] Connection header received
0812 05:29:50.935325 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:50.935326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:50.935341 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:50.935412 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:50.936246 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.936780 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.938085 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.938308 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.938467 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:50.941030 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:50.941032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:50.941033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:50.941042 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:50.942714 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:50.943242 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:50.943243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:50.943243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:50.943296 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:50.943600 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:50.943601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:50.943601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:50.943824 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:50.943896 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:50.944244 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 05:29:50.944401 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 05:29:50.944667 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.934479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38491 (local address 127.0.0.1:50814)
0812 05:29:50.934732 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.934772 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:50.935229 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:50.935484 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:50.935487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:50.935813 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:50.936034 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:50.936038 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.936900 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.936901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:50.937986 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:50.937989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.938496 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.938498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:50.938615 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:50.939277 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:50.939289 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:50.940925 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:50.942797 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:50.942806 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:50.942809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:50.943162 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:50.943364 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:50.943365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:50.943366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:50.943526 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:50.943902 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:50.943909 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:50.944041 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 05:29:50.944177 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 05:29:50.944380 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.24.66.124@KRBTEST.COM; defaulting to no policy
I20260812 05:29:50.949491 16496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.66.124@KRBTEST.COM" created.
Aug 12 05:29:50 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:29:50.958889 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.949779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:42767 (local address 127.0.0.1:36874)
0812 05:29:50.949851 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.949862 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:50.949909 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:50.950370 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:50.950371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:50.950519 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:50.950572 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:50.950573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.951480 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.951481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:50.951988 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:50.951989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.952666 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.952669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:50.952789 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:50.953348 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:50.953354 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:50.954920 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:50.957370 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:50.957371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:50.957372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:50.957623 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:50.957894 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:50.957896 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:50.957896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:50.957977 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:50.958383 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:50.958384 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:50.958443 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 05:29:50.958532 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 05:29:50.958708 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 05:29:50.958972 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.949934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.24.66.125:42767)
0812 05:29:50.950148 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:50.950155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:50.950177 (+ 22us) server_negotiation.cc:407] Connection header received
0812 05:29:50.950218 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:50.950220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:50.950240 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:50.950358 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:50.950665 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.951378 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.952073 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.952570 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.952650 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:50.955019 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:50.955032 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:50.955033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:50.955044 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:50.957264 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:50.957732 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:50.957736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:50.957736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:50.957812 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:50.958054 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:50.958055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:50.958056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:50.958281 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:50.958417 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:50.958596 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 05:29:50.958798 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":182,"thread_start_us":93,"threads_started":1}
I20260812 05:29:50.959751 16496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:50.960188 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.960017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43520)
0812 05:29:50.960083 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.960157 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 05:29:50.960438 16439 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
I20260812 05:29:50.960963 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.960842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43526)
0812 05:29:50.960900 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.960937 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 05:29:50.961184 16496 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:42277: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.66.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab.
Entry for principal kudu/127.24.66.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab.
I20260812 05:29:50.970502 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_bqoao/build/release/bin/kudu
/tmp/dist-test-task_bqoao/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.124:42277
--webserver_interface=127.24.66.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab
--principal=kudu/127.24.66.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.66.84:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:38491,127.24.66.125:42767,127.24.66.124:42277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 05:29:50.997217 16412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } attempt: 3
Aug 12 05:29:50 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:29:51.004887 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.997674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:42767 (local address 127.0.0.1:36886)
0812 05:29:50.997742 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:50.997759 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:50.997830 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:50.998008 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:50.998009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:50.998186 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:50.998264 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:50.998267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.999011 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.999012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:50.999560 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:50.999562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.999968 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.999969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:51.000054 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.000598 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:51.000603 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:51.001907 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:51.003822 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.003823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.003823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.004004 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.004228 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.004228 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.004229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.004293 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.004599 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:51.004600 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:51.004647 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 05:29:51.004700 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 05:29:51.004810 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 05:29:51.004940 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:50.997782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36886 (local address 127.24.66.125:42767)
0812 05:29:50.997871 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:50.997873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:50.997884 (+ 11us) server_negotiation.cc:407] Connection header received
0812 05:29:50.997913 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:50.997914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:50.997931 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:50.998000 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:50.998406 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.998920 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.999657 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:50.999889 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:50.999970 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.002007 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.002009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.002009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.002019 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.003718 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.004081 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.004082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.004082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.004150 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.004362 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.004362 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.004363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.004513 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.004599 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.004750 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.004846 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 05:29:51.005434 16412 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:51.005847 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.005725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43538)
0812 05:29:51.005785 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.005817 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 05:29:51.006291 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.006187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43546)
0812 05:29:51.006242 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.006270 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 05:29:51.006419 16412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:42277: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
I20260812 05:29:51.010452 16496 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } attempt: 1
I20260812 05:29:51.010986 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.010817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43560)
0812 05:29:51.010891 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.010941 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 05:29:51.011430 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.011289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43564)
0812 05:29:51.011362 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.011397 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 05:29:51.011545 16496 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:42277: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
I20260812 05:29:51.037586 16412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } attempt: 1
I20260812 05:29:51.038180 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.038015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43572)
0812 05:29:51.038101 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.038148 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 05:29:51.038713 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.038613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43582)
0812 05:29:51.038662 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.038693 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 05:29:51.038854 16412 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:42277: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
W20260812 05:29:51.061194 16504 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 05:29:51.061589 16504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 05:29:51.061719 16504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 05:29:51.061761 16504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 05:29:51.063225 16504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 05:29:51.063301 16504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 05:29:51.063329 16504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 05:29:51.063362 16504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 05:29:51.063390 16504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 05:29:51.065024 16504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:38491,127.24.66.125:42767,127.24.66.124:42277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/krb5kdc/kudu_127.24.66.124.keytab
--principal=kudu/127.24.66.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.124:42277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.66.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 04:20:34 UTC on ccycloud.quasar-phqdqp.root.comops.site
build id 4581871
I20260812 05:29:51.065336 16504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 05:29:51.065590 16504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 05:29:51.070787 16511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:51.070811 16513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 05:29:51.070811 16510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 05:29:51.071157 16504 server_base.cc:1034] running on GCE node
I20260812 05:29:51.071403 16504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 05:29:51.071668 16504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 05:29:51.072885 16504 hybrid_clock.cc:648] HybridClock initialized: now 1786512591072886 us; error 46 us; skew 500 ppm
Aug 12 05:29:51 dist-test-slave-zpfg krb5kdc[16325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512591, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 05:29:51.076108 16504 init.cc:377] Logged in from keytab as kudu/127.24.66.124@KRBTEST.COM (short username kudu)
I20260812 05:29:51.076659 16504 webserver.cc:460] Webserver started at http://127.24.66.124:34891/ using document root <none> and password file <none>
I20260812 05:29:51.076871 16504 fs_manager.cc:359] Metadata directory not provided
I20260812 05:29:51.076913 16504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 05:29:51.077013 16504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 05:29:51.078095 16504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/data/instance:
uuid: "d4d4e84822944d639fab5413e6c07db0"
format_stamp: "Formatted at 2026-08-12 05:29:51 on dist-test-slave-zpfg"
server_key: "ed97a345ba738d8a2d767b751186e9ab"
server_key_iv: "e5455cf7a54578f166da1be33deb7d4a"
server_key_version: "encryptionkey@0"
I20260812 05:29:51.078497 16504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/wal/instance:
uuid: "d4d4e84822944d639fab5413e6c07db0"
format_stamp: "Formatted at 2026-08-12 05:29:51 on dist-test-slave-zpfg"
server_key: "ed97a345ba738d8a2d767b751186e9ab"
server_key_iv: "e5455cf7a54578f166da1be33deb7d4a"
server_key_version: "encryptionkey@0"
I20260812 05:29:51.079707 16496 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } attempt: 2
I20260812 05:29:51.080010 16504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 05:29:51.080204 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.080032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43594)
0812 05:29:51.080117 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.080169 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 05:29:51.080669 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.080581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43596)
0812 05:29:51.080625 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.080651 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 05:29:51.080833 16496 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.66.124:42277: Network error: Client connection negotiation failed: client connection to 127.24.66.124:42277: connect: Connection refused (error 111)
I20260812 05:29:51.081045 16520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 05:29:51.081336 16504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 05:29:51.081442 16504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/data,/tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/wal
uuid: "d4d4e84822944d639fab5413e6c07db0"
format_stamp: "Formatted at 2026-08-12 05:29:51 on dist-test-slave-zpfg"
server_key: "ed97a345ba738d8a2d767b751186e9ab"
server_key_iv: "e5455cf7a54578f166da1be33deb7d4a"
server_key_version: "encryptionkey@0"
I20260812 05:29:51.081532 16504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 05:29:51.093207 16504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 05:29:51.095860 16504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 05:29:51.096046 16504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 05:29:51.100929 16504 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.124:42277
I20260812 05:29:51.100973 16576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.124:42277 every 8 connection(s)
I20260812 05:29:51.101372 16504 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/data/info.pb
I20260812 05:29:51.102618 16577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 05:29:51.104331 16577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:51.107028 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_bqoao/build/release/bin/kudu as pid 16504
I20260812 05:29:51.107148 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_bqoao/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786512301961891-24841-0/minicluster-data/master-2/wal/instance
I20260812 05:29:51.107359 24841 external_mini_cluster.cc:1428] Setting key c7bd896f9059a7a0075c515f3bacc381
Aug 12 05:29:51 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512591, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
Aug 12 05:29:51 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
I20260812 05:29:51.114009 16412 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } attempt: 2
I20260812 05:29:51.115186 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.105067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.126:38491 (local address 127.0.0.1:50828)
0812 05:29:51.105257 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.105302 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:51.105803 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:51.106036 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:51.106039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:51.106306 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:51.106631 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:51.106637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.107824 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.107825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:51.108708 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:51.108711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.109135 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.109136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:51.109249 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.109893 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:51.109902 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:51.111566 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:51.113357 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.113365 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.113367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.113586 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.113871 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.113871 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.113872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.114062 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.114414 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:51.114423 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:51.114572 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 05:29:51.114698 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 05:29:51.114884 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":107,"threads_started":1}
I20260812 05:29:51.115442 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.105085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.24.66.126:38491)
0812 05:29:51.105165 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.105167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.105400 (+ 233us) server_negotiation.cc:407] Connection header received
0812 05:29:51.105894 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.105896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.105906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.105971 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.106875 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.107682 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.108809 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.109009 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.109097 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.111684 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.111686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.111686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.111701 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.113259 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.113674 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.113676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.113676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.113788 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.114122 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.114124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.114124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.114297 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.114420 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.114723 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.115346 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 05:29:51.115913 16577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:51.118324 16581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.107789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43600 (local address 127.24.66.124:42277)
0812 05:29:51.108114 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.108119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.108137 (+ 18us) server_negotiation.cc:407] Connection header received
0812 05:29:51.108155 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.108156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.108172 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.108255 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.108894 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.109711 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.110593 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.110903 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.110995 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.113631 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.113639 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.113640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.113646 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.115969 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.116396 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.116399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.116400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.116455 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.117331 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.117332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.117332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.117490 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.118089 (+ 599us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.118143 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.118224 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":120,"threads_started":1}
Aug 12 05:29:51 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.126@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
Aug 12 05:29:51 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512591, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:29:51.124267 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.116201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.125:42767 (local address 127.0.0.1:36894)
0812 05:29:51.116240 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.116250 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:51.116284 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:51.116412 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:51.116413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:51.116559 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:51.116612 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:51.116614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.117392 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.117393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:51.118112 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:51.118115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.118651 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.118652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:51.118730 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.119150 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:51.119154 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:51.120573 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:51.122953 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.122954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.122954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.123288 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.123538 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.123539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.123539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.123599 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.123939 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:51.123940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:51.123986 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 05:29:51.124057 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 05:29:51.124153 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 05:29:51.124336 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.116263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36894 (local address 127.24.66.125:42767)
0812 05:29:51.116317 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.116318 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.116327 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:29:51.116351 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.116352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.116360 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.116411 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.116700 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.117183 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.118207 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.118556 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.118635 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.120664 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.120665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.120666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.120673 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.122868 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.123372 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.123373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.123374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.123443 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.123673 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.123674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.123674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.123833 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.123880 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.124123 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.124210 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 05:29:51 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.126@KRBTEST.COM
I20260812 05:29:51.124621 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.114265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43606)
0812 05:29:51.114327 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.114371 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:51.114413 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:51.114688 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:51.114689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:51.115144 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:51.115192 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:51.115195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.116508 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.116510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:51.117090 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:51.117092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.117553 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.117554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:51.117616 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.118062 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:51.118067 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:51.119706 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:51.123416 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.123417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.123418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.123599 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.123825 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.123826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.123826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.123885 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.124311 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:51.124312 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:51.124356 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 05:29:51.124407 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 05:29:51.124506 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 05:29:51.124639 16577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } has no permanent_uuid. Determining permanent_uuid...
I20260812 05:29:51.125109 16582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.114372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43606 (local address 127.24.66.124:42277)
0812 05:29:51.114515 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.114517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.114524 (+ 7us) server_negotiation.cc:407] Connection header received
0812 05:29:51.114545 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.114545 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.114558 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.114785 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.115691 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.116444 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.117171 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.117472 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.117537 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.119804 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.119806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.119806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.119815 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.123321 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.123677 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.123678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.123678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.123754 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.124088 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.124090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.124090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.124223 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.124818 (+ 595us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.124860 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.124952 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260812 05:29:51.127394 16412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994: Bootstrap starting.
I20260812 05:29:51.128115 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.119546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.24.66.126:38491)
0812 05:29:51.119646 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.119648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.119656 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:29:51.119678 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.119679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.119690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.119742 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.120333 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.120921 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.121970 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.122282 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.122418 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.125338 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.125340 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.125340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.125348 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.126914 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.127256 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.127257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.127258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.127352 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.127556 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.127557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.127557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.127686 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.127863 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.127946 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.128025 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 05:29:51.128243 16412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:51.128755 16412 log.cc:824] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:51.129695 16412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994: No bootstrap required, opened a new log
Aug 12 05:29:51 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512591, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.124@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
I20260812 05:29:51.132105 16412 raft_consensus.cc:348] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } }
I20260812 05:29:51.132296 16412 raft_consensus.cc:729] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c551664f2e22459896ae6ab0cd1fe994, State: Initialized, Role: FOLLOWER
I20260812 05:29:51.132401 16412 consensus_queue.cc:260] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } }
I20260812 05:29:51.132728 16584 sys_catalog.cc:455] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } } }
I20260812 05:29:51.132838 16412 sys_catalog.cc:564] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:51.132937 16584 sys_catalog.cc:458] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [sys.catalog]: This master's current role is: FOLLOWER
Aug 12 05:29:51 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.66.125@KRBTEST.COM
I20260812 05:29:51.133754 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.124881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43622)
0812 05:29:51.125012 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.125022 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:51.125066 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:51.125359 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:51.125360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:51.125532 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:51.125571 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:51.125573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.127607 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.127608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:51.128383 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:51.128386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.128851 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.128852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:51.128904 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.129286 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:51.129290 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:51.130724 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:51.132592 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.132593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.132593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.132804 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.133001 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.133002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.133002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.133053 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.133398 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:51.133399 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:51.133444 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 05:29:51.133497 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 05:29:51.133632 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260812 05:29:51.134248 16581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.124951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43622 (local address 127.24.66.124:42277)
0812 05:29:51.124997 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.124999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.125056 (+ 57us) server_negotiation.cc:407] Connection header received
0812 05:29:51.125132 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.125133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.125147 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.125321 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.125635 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.127535 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.128471 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.128752 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.128832 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.130816 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.130818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.130818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.130824 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.132498 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.132887 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.132888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.132888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.132937 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.133120 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.133121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.133122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.133315 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.133970 (+ 655us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.134013 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.134106 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:29:51.135866 16577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0: Bootstrap starting.
I20260812 05:29:51.136698 16577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:51.137431 16577 log.cc:824] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:51.137567 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.128979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36908 (local address 127.24.66.125:42767)
0812 05:29:51.129302 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.129305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.129315 (+ 10us) server_negotiation.cc:407] Connection header received
0812 05:29:51.129338 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.129339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.129349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.129426 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.129932 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.130471 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.131178 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.131466 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.131502 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.134177 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.134179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.134179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.134187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.135982 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.136461 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.136463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.136463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.136532 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.136838 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.136840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.136840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.136967 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.137346 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.137396 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.137479 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 05:29:51.135629 16596 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:51.138423 16596 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:51.138506 16577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0: No bootstrap required, opened a new log
I20260812 05:29:51.139768 24841 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 05:29:51.140846 16385 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c551664f2e22459896ae6ab0cd1fe994"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50832
I20260812 05:29:51.140986 16385 raft_consensus.cc:482] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 05:29:51.140879 16577 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } }
I20260812 05:29:51.141036 16385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:51.141049 16577 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d4e84822944d639fab5413e6c07db0, State: Initialized, Role: FOLLOWER
I20260812 05:29:51.141150 16577 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } }
I20260812 05:29:51.141448 16601 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } } }
I20260812 05:29:51.141516 16577 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:51.141597 16601 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:29:51.142026 16385 raft_consensus.cc:504] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } }
I20260812 05:29:51.139232 16339 master_runner.cc:186] Error getting master registration for 127.24.66.125:42767: OK, instance_id { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" instance_seqno: 1786512590928918 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:29:51.142248 16339 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 05:29:51.142297 16385 leader_election.cc:288] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [CANDIDATE]: Term 1 election: Requested vote from peers c815b72bb62d4b7e9d78fec26f9d5526 (127.24.66.125:42767), d4d4e84822944d639fab5413e6c07db0 (127.24.66.124:42277)
I20260812 05:29:51.142758 16547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c551664f2e22459896ae6ab0cd1fe994" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d4d4e84822944d639fab5413e6c07db0"
I20260812 05:29:51.142886 16547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:51.143842 16547 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c551664f2e22459896ae6ab0cd1fe994 in term 1.
I20260812 05:29:51.144352 16357 leader_election.cc:302] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c551664f2e22459896ae6ab0cd1fe994, d4d4e84822944d639fab5413e6c07db0; no voters:
I20260812 05:29:51.144577 16584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 05:29:51.144790 16584 raft_consensus.cc:686] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [term 1 LEADER]: Becoming Leader. State: Replica: c551664f2e22459896ae6ab0cd1fe994, State: Running, Role: LEADER
I20260812 05:29:51.144898 16584 consensus_queue.cc:237] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } }
W20260812 05:29:51.145056 16617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:51.145148 16617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:51.145351 16584 sys_catalog.cc:455] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c551664f2e22459896ae6ab0cd1fe994. Latest consensus state: current_term: 1 leader_uuid: "c551664f2e22459896ae6ab0cd1fe994" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } } }
I20260812 05:29:51.145452 16584 sys_catalog.cc:458] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:51.145911 16623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 05:29:51.146128 16623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 05:29:51.147744 16547 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [term 1 FOLLOWER]: Refusing update from remote peer c551664f2e22459896ae6ab0cd1fe994: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:51.148031 16584 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:51.146986 16504 master_runner.cc:186] Error getting master registration for 127.24.66.125:42767: OK, instance_id { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" instance_seqno: 1786512590928918 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 05:29:51.149461 16504 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 05:29:51.150408 16601 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c551664f2e22459896ae6ab0cd1fe994. Latest consensus state: current_term: 1 leader_uuid: "c551664f2e22459896ae6ab0cd1fe994" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } } }
I20260812 05:29:51.150508 16601 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:29:51.152829 16581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.143921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43638 (local address 127.24.66.124:42277)
0812 05:29:51.143973 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.143976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.144768 (+ 792us) server_negotiation.cc:407] Connection header received
0812 05:29:51.144796 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.144797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.144827 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.144888 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.145894 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.146605 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.147721 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.148015 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.148060 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.149836 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.149837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.149838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.149844 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.151569 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.151947 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.151948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.151949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.152024 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.152252 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.152253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.152254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.152405 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.152497 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.152628 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.152721 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 05:29:51.159929 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.144160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.24.66.126:38491)
0812 05:29:51.144225 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.144228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.148562 (+ 4334us) server_negotiation.cc:407] Connection header received
0812 05:29:51.148704 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.148705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.148715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.148742 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.151127 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.151867 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.152863 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.153155 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.153197 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.155373 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.155375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.155375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.155383 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.157648 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.158281 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.158282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.158294 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.158369 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.158670 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.158671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.158671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.158825 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.158921 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.159460 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.159822 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 05:29:51.162434 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.144522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36924 (local address 127.24.66.125:42767)
0812 05:29:51.144568 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.144570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.152910 (+ 8340us) server_negotiation.cc:407] Connection header received
0812 05:29:51.152937 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.152938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.152950 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.153000 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.154099 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.154849 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.156384 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.156699 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.156746 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.158718 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.158719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.158720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.158728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.160438 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.161100 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.161102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.161102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.161166 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.161609 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.161609 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.161610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.161738 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.161824 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.162200 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.162290 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 05:29:51.163350 16584 sys_catalog.cc:455] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c551664f2e22459896ae6ab0cd1fe994" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } } }
I20260812 05:29:51.163455 16584 sys_catalog.cc:458] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:51.163609 16601 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c551664f2e22459896ae6ab0cd1fe994" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } } }
I20260812 05:29:51.163765 16601 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:29:51.163786 16624 mvcc.cc:205] Tried to move back new op lower bound from 7317555573338554368 to 7317555573330186240. Current Snapshot: MvccSnapshot[applied={T|T < 7317555573338554368}]
I20260812 05:29:51.163906 16625 mvcc.cc:205] Tried to move back new op lower bound from 7317555573338554368 to 7317555573330186240. Current Snapshot: MvccSnapshot[applied={T|T < 7317555573338554368}]
I20260812 05:29:51.171295 16623 catalog_manager.cc:1347] Generated new cluster ID: 837e63fb31184178ad53be346645d9db
I20260812 05:29:51.171391 16623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 05:29:51.171989 16496 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } attempt: 3
Aug 12 05:29:51 dist-test-slave-zpfg krb5kdc[16325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786512590, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.66.125@KRBTEST.COM for kudu/127.24.66.124@KRBTEST.COM
I20260812 05:29:51.180365 16581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.172330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43650 (local address 127.24.66.124:42277)
0812 05:29:51.172368 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.172370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.172379 (+ 9us) server_negotiation.cc:407] Connection header received
0812 05:29:51.172456 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.172457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.172467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.172529 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.172895 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.173677 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.174628 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.174931 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.175013 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.177190 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 05:29:51.177191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 05:29:51.177192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 05:29:51.177200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 05:29:51.179161 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.179543 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.179544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.179545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.179607 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 05:29:51.179805 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 05:29:51.179806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 05:29:51.179807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 05:29:51.179942 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 05:29:51.180005 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:51.180188 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 05:29:51.180270 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 05:29:51.180382 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:51.172289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.66.124:42277 (local address 127.0.0.1:43650)
0812 05:29:51.172337 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 05:29:51.172349 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 05:29:51.172388 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 05:29:51.172582 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 05:29:51.172584 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 05:29:51.172773 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 05:29:51.172820 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:51.172822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.173758 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.173759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:51.174545 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 05:29:51.174547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 05:29:51.175010 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 05:29:51.175011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 05:29:51.175064 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:51.175548 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 05:29:51.175553 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 05:29:51.177094 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 05:29:51.179240 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.179242 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.179242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.179464 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.179667 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 05:29:51.179668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 05:29:51.179668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 05:29:51.179733 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 05:29:51.179999 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 05:29:51.180014 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 05:29:51.180078 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 05:29:51.180166 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 05:29:51.180278 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 05:29:51.182381 16496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526: Bootstrap starting.
I20260812 05:29:51.183254 16496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526: Neither blocks nor log segments found. Creating new log.
I20260812 05:29:51.183614 16496 log.cc:824] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526: Log is configured to *not* fsync() on all Append() calls
I20260812 05:29:51.185312 16496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526: No bootstrap required, opened a new log
I20260812 05:29:51.189363 16496 raft_consensus.cc:348] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } }
I20260812 05:29:51.189993 16496 raft_consensus.cc:729] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c815b72bb62d4b7e9d78fec26f9d5526, State: Initialized, Role: FOLLOWER
I20260812 05:29:51.190308 16496 consensus_queue.cc:260] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } }
I20260812 05:29:51.191344 16467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [term 0 FOLLOWER]: Advancing to term 1
I20260812 05:29:51.192584 16467 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [term 1 FOLLOWER]: Refusing update from remote peer c551664f2e22459896ae6ab0cd1fe994: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 05:29:51.192966 16620 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 05:29:51.191370 16468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c551664f2e22459896ae6ab0cd1fe994" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c815b72bb62d4b7e9d78fec26f9d5526"
I20260812 05:29:51.193823 16468 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c551664f2e22459896ae6ab0cd1fe994 in term 1.
I20260812 05:29:51.191251 16496 sys_catalog.cc:564] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 05:29:51.197244 16630 sys_catalog.cc:455] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c551664f2e22459896ae6ab0cd1fe994" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } } }
I20260812 05:29:51.197359 16630 sys_catalog.cc:458] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:29:51.197414 16630 sys_catalog.cc:455] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c551664f2e22459896ae6ab0cd1fe994. Latest consensus state: current_term: 1 leader_uuid: "c551664f2e22459896ae6ab0cd1fe994" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } } }
I20260812 05:29:51.197464 16630 sys_catalog.cc:458] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 05:29:51.197494 16630 sys_catalog.cc:455] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c551664f2e22459896ae6ab0cd1fe994" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } } }
I20260812 05:29:51.197538 16630 sys_catalog.cc:458] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 05:29:51.198421 16646 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 05:29:51.198483 16646 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 05:29:51.198726 16623 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 05:29:51.199555 16623 catalog_manager.cc:1504] Loading token signing keys...
I20260812 05:29:51.203681 16634 mvcc.cc:205] Tried to move back new op lower bound from 7317555573338554368 to 7317555573330186240. Current Snapshot: MvccSnapshot[applied={T|T < 7317555573338554368}]
I20260812 05:29:51.203650 16584 sys_catalog.cc:455] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c551664f2e22459896ae6ab0cd1fe994" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c551664f2e22459896ae6ab0cd1fe994" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 38491 } } peers { permanent_uuid: "c815b72bb62d4b7e9d78fec26f9d5526" member_type: VOTER last_known_addr { host: "127.24.66.125" port: 42767 } } peers { permanent_uuid: "d4d4e84822944d639fab5413e6c07db0" member_type: VOTER last_known_addr { host: "127.24.66.124" port: 42277 } } }
I20260812 05:29:51.203775 16584 sys_catalog.cc:458] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994 [sys.catalog]: This master's current role is: LEADER
I20260812 05:29:51.235128 16623 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c551664f2e22459896ae6ab0cd1fe994: Generated new TSK 0
I20260812 05:29:51.235381 16623 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 05:29:51.269333 16581 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.267509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.24.66.124:42277)
0812 05:29:51.267558 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.267560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.268565 (+ 1005us) server_negotiation.cc:407] Connection header received
0812 05:29:51.268675 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.268676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.268689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.268720 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.269249 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43652: BlockingRecv error: recv got EOF from 127.0.0.1:43652 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 05:29:51.270135 16415 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.267915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50858 (local address 127.24.66.126:38491)
0812 05:29:51.267959 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.267961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.269592 (+ 1631us) server_negotiation.cc:407] Connection header received
0812 05:29:51.269694 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.269696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.269709 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.269739 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.270054 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50858: BlockingRecv error: recv got EOF from 127.0.0.1:50858 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 05:29:51.270924 16501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.268287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36934 (local address 127.24.66.125:42767)
0812 05:29:51.268329 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.268331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.270388 (+ 2057us) server_negotiation.cc:407] Connection header received
0812 05:29:51.270489 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.270490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.270501 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.270531 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.270847 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36934: BlockingRecv error: recv got EOF from 127.0.0.1:36934 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 05:29:51.273451 16501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.272625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36948 (local address 127.24.66.125:42767)
0812 05:29:51.272678 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.272680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.272888 (+ 208us) server_negotiation.cc:407] Connection header received
0812 05:29:51.272994 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.272995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.273016 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.273047 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.273400 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36948: BlockingRecv error: recv got EOF from 127.0.0.1:36948 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 05:29:51.276777 16501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.275881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36964 (local address 127.24.66.125:42767)
0812 05:29:51.275918 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.275920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.276206 (+ 286us) server_negotiation.cc:407] Connection header received
0812 05:29:51.276312 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.276313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.276326 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.276356 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.276723 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36964: BlockingRecv error: recv got EOF from 127.0.0.1:36964 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 05:29:51.281206 16501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.280310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.24.66.125:42767)
0812 05:29:51.280358 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.280360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.280604 (+ 244us) server_negotiation.cc:407] Connection header received
0812 05:29:51.280723 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.280724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.280735 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.280765 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.281145 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36978: BlockingRecv error: recv got EOF from 127.0.0.1:36978 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 05:29:51.287552 16501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.286621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.24.66.125:42767)
0812 05:29:51.286664 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.286666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.286917 (+ 251us) server_negotiation.cc:407] Connection header received
0812 05:29:51.287022 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.287023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.287034 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.287065 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.287476 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36988: BlockingRecv error: recv got EOF from 127.0.0.1:36988 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 05:29:51.298053 16501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.297052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.24.66.125:42767)
0812 05:29:51.297099 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.297102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.297392 (+ 290us) server_negotiation.cc:407] Connection header received
0812 05:29:51.297506 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.297507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.297519 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.297551 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.297980 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36992: BlockingRecv error: recv got EOF from 127.0.0.1:36992 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 05:29:51.316795 16501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.315723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37004 (local address 127.24.66.125:42767)
0812 05:29:51.315769 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.315771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.316093 (+ 322us) server_negotiation.cc:407] Connection header received
0812 05:29:51.316237 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.316238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.316251 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.316284 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.316711 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37004: BlockingRecv error: recv got EOF from 127.0.0.1:37004 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 05:29:51.351215 16501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.350385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.24.66.125:42767)
0812 05:29:51.350433 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.350435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.350576 (+ 141us) server_negotiation.cc:407] Connection header received
0812 05:29:51.350630 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.350631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.350643 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.350722 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.351152 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37008: BlockingRecv error: recv got EOF from 127.0.0.1:37008 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 05:29:51.417873 16501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.416959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.24.66.125:42767)
0812 05:29:51.417038 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.417040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.417190 (+ 150us) server_negotiation.cc:407] Connection header received
0812 05:29:51.417299 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.417301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.417315 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.417384 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.417788 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37022: BlockingRecv error: recv got EOF from 127.0.0.1:37022 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 05:29:51.548071 16501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.547253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37026 (local address 127.24.66.125:42767)
0812 05:29:51.547328 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.547330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.547390 (+ 60us) server_negotiation.cc:407] Connection header received
0812 05:29:51.547453 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.547454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.547464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.547533 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.547977 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37026: BlockingRecv error: recv got EOF from 127.0.0.1:37026 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 05:29:51.806555 16501 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 05:29:51.805680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37028 (local address 127.24.66.125:42767)
0812 05:29:51.805768 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:51.805770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:51.805817 (+ 47us) server_negotiation.cc:407] Connection header received
0812 05:29:51.805883 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:51.805884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:51.805895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:51.805943 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 05:29:51.806487 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37028: BlockingRecv error: recv got EOF from 127.0.0.1:37028 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 05:29:52.145634 16617 catalog_manager.cc:1259] Loaded cluster ID: 837e63fb31184178ad53be346645d9db
I20260812 05:29:52.145735 16617 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0: loading cluster ID for follower catalog manager: success
I20260812 05:29:52.146763 16617 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0: acquiring CA information for follower catalog manager: success
I20260812 05:29:52.147130 16617 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d4d4e84822944d639fab5413e6c07db0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:29:52.198942 16646 catalog_manager.cc:1259] Loaded cluster ID: 837e63fb31184178ad53be346645d9db
I20260812 05:29:52.199025 16646 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526: loading cluster ID for follower catalog manager: success
I20260812 05:29:52.199880 16646 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526: acquiring CA information for follower catalog manager: success
I20260812 05:29:52.200199 16646 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c815b72bb62d4b7e9d78fec26f9d5526: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 05:29:52.322520 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:52.319865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37030 (local address 127.24.66.125:42767)
0812 05:29:52.319975 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:52.319977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:52.319995 (+ 18us) server_negotiation.cc:407] Connection header received
0812 05:29:52.320029 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:52.320030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:52.320085 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:52.320178 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:52.320357 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:52.320867 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:52.321648 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:52.321861 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:52.321919 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:52.322069 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:52.322199 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:52.322266 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:52.322325 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 05:29:52.322400 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260812 05:29:52.327080 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:52.324495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43668 (local address 127.24.66.124:42277)
0812 05:29:52.324624 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:52.324626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:52.324634 (+ 8us) server_negotiation.cc:407] Connection header received
0812 05:29:52.324674 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:52.324675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:52.324713 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:52.324806 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:52.324947 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:52.325534 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:52.326264 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:52.326560 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:52.326622 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:52.326766 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:52.326882 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:52.326941 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:52.326993 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 05:29:52.327012 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260812 05:29:52.331872 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:52.328974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43672 (local address 127.24.66.124:42277)
0812 05:29:52.329041 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:52.329043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:52.329174 (+ 131us) server_negotiation.cc:407] Connection header received
0812 05:29:52.329204 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:52.329204 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:52.329216 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:52.329268 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:52.329375 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:52.329976 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:52.331009 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:52.331299 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:52.331385 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:52.331544 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:52.331575 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:52.331622 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:52.331678 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 05:29:52.331699 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 05:29:52.332180 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:52.329146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37036 (local address 127.24.66.125:42767)
0812 05:29:52.329213 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:52.329215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:52.329418 (+ 203us) server_negotiation.cc:407] Connection header received
0812 05:29:52.329506 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:52.329507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:52.329518 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:52.329567 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:52.329740 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:52.330281 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:52.331266 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:52.331587 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:52.331658 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:52.331853 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:52.331885 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:52.331935 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:52.332052 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 05:29:52.332081 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 05:29:52.333464 16736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 05:29:52.328889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50860 (local address 127.24.66.126:38491)
0812 05:29:52.329007 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 05:29:52.329009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 05:29:52.329423 (+ 414us) server_negotiation.cc:407] Connection header received
0812 05:29:52.329572 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 05:29:52.329573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 05:29:52.329638 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 05:29:52.329749 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 05:29:52.330610 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:52.331174 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:52.332454 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 05:29:52.332815 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 05:29:52.332863 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 05:29:52.333105 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 05:29:52.333205 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 05:29:52.333257 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 05:29:52.333307 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 05:29:52.333326 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":61,"threads_started":1}
I20260812 05:29:52.335417 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 16339
I20260812 05:29:52.343863 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 16420
I20260812 05:29:52.351379 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_bqoao/build/release/bin/kudu with pid 16504
2026-08-12T05:29:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2988 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6061 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (290398 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST